-- Logs begin at Tue 2022-08-30 15:16:41 UTC, end at Tue 2022-08-30 16:20:11 UTC. -- Aug 30 15:45:03.654536 dv-cinder-855058-1-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Aug 30 15:45:03.661895 dv-cinder-855058-1-iscsi devstack@keystone.service[81499]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Aug 30 15:45:03.663914 dv-cinder-855058-1-iscsi devstack@keystone.service[81499]: *** Starting uWSGI 2.0.18-debian (64bit) on [Tue Aug 30 15:45:03 2022] *** Aug 30 15:45:03.663914 dv-cinder-855058-1-iscsi devstack@keystone.service[81499]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Aug 30 15:45:03.663914 dv-cinder-855058-1-iscsi devstack@keystone.service[81499]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Aug 30 15:45:03.663914 dv-cinder-855058-1-iscsi devstack@keystone.service[81499]: nodename: dv-cinder-855058-1-iscsi Aug 30 15:45:03.663914 dv-cinder-855058-1-iscsi devstack@keystone.service[81499]: machine: x86_64 Aug 30 15:45:03.663914 dv-cinder-855058-1-iscsi devstack@keystone.service[81499]: clock source: unix Aug 30 15:45:03.663914 dv-cinder-855058-1-iscsi devstack@keystone.service[81499]: pcre jit disabled Aug 30 15:45:03.664333 dv-cinder-855058-1-iscsi devstack@keystone.service[81499]: detected number of CPU cores: 4 Aug 30 15:45:03.664333 dv-cinder-855058-1-iscsi devstack@keystone.service[81499]: current working directory: / Aug 30 15:45:03.664333 dv-cinder-855058-1-iscsi devstack@keystone.service[81499]: detected binary path: /usr/bin/uwsgi-core Aug 30 15:45:03.664333 dv-cinder-855058-1-iscsi devstack@keystone.service[81499]: your processes number limit is 47680 Aug 30 15:45:03.664333 dv-cinder-855058-1-iscsi devstack@keystone.service[81499]: your memory page size is 4096 bytes Aug 30 15:45:03.664333 dv-cinder-855058-1-iscsi devstack@keystone.service[81499]: detected max file descriptor number: 2048 Aug 30 15:45:03.664333 dv-cinder-855058-1-iscsi devstack@keystone.service[81499]: lock engine: pthread robust mutexes Aug 30 15:45:03.664333 dv-cinder-855058-1-iscsi devstack@keystone.service[81499]: thunder lock: enabled Aug 30 15:45:03.664333 dv-cinder-855058-1-iscsi devstack@keystone.service[81499]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Aug 30 15:45:03.664333 dv-cinder-855058-1-iscsi devstack@keystone.service[81499]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Aug 30 15:45:03.704177 dv-cinder-855058-1-iscsi devstack@keystone.service[81499]: Python main interpreter initialized at 0x55adf673e9c0 Aug 30 15:45:03.704177 dv-cinder-855058-1-iscsi devstack@keystone.service[81499]: python threads support enabled Aug 30 15:45:03.704177 dv-cinder-855058-1-iscsi devstack@keystone.service[81499]: your server socket listen backlog is limited to 100 connections Aug 30 15:45:03.704177 dv-cinder-855058-1-iscsi devstack@keystone.service[81499]: your mercy for graceful operations on workers is 90 seconds Aug 30 15:45:03.704543 dv-cinder-855058-1-iscsi devstack@keystone.service[81499]: mapped 403077 bytes (393 KB) for 2 cores Aug 30 15:45:03.704643 dv-cinder-855058-1-iscsi devstack@keystone.service[81499]: *** Operational MODE: preforking *** Aug 30 15:45:03.704920 dv-cinder-855058-1-iscsi devstack@keystone.service[81499]: *** uWSGI is running in multiple interpreter mode *** Aug 30 15:45:03.704920 dv-cinder-855058-1-iscsi devstack@keystone.service[81499]: spawned uWSGI master process (pid: 81499) Aug 30 15:45:03.705177 dv-cinder-855058-1-iscsi systemd[1]: Started Devstack devstack@keystone.service. Aug 30 15:45:03.707835 dv-cinder-855058-1-iscsi devstack@keystone.service[81499]: spawned uWSGI worker 1 (pid: 81511, cores: 1) Aug 30 15:45:03.707835 dv-cinder-855058-1-iscsi devstack@keystone.service[81499]: spawned uWSGI worker 2 (pid: 81512, cores: 1) Aug 30 15:45:03.707835 dv-cinder-855058-1-iscsi devstack@keystone.service[81499]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Aug 30 15:45:07.931314 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 30 15:45:07.931314 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: from cryptography.utils import int_from_bytes Aug 30 15:45:07.932131 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 30 15:45:07.932131 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: from cryptography.utils import int_from_bytes Aug 30 15:45:07.932247 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 30 15:45:07.932247 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: from cryptography.utils import int_from_bytes Aug 30 15:45:07.932656 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 30 15:45:07.932656 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: from cryptography.utils import int_from_bytes Aug 30 15:45:07.981079 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 30 15:45:07.981307 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 30 15:45:07.981465 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] command line args: [] {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 30 15:45:07.981701 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 30 15:45:07.981871 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] ================================================================================ {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 30 15:45:07.981961 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 30 15:45:07.982080 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 30 15:45:07.982166 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] admin_token = **** {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 15:45:07.982305 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] command line args: [] {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 30 15:45:07.982441 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] config_dir = [] {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 15:45:07.982508 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 15:45:07.982584 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 30 15:45:07.982651 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] config_source = [] {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 15:45:07.982714 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] ================================================================================ {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 30 15:45:07.982868 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] debug = True {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 15:45:07.982947 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] admin_token = **** {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 15:45:07.983007 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 15:45:07.985158 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] config_dir = [] {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 15:45:07.985253 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 15:45:07.985332 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 15:45:07.985400 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 15:45:07.985458 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] config_source = [] {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 15:45:07.985458 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] debug = True {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 15:45:07.985575 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 15:45:07.985633 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 15:45:07.985723 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 15:45:07.985783 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 15:45:07.985851 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] list_limit = None {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 15:45:07.985915 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 15:45:07.985974 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] log_config_append = None {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 15:45:07.985974 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 15:45:07.986087 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 15:45:07.986150 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] log_dir = None {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 15:45:07.986215 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 15:45:07.986274 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] log_file = None {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 15:45:07.986335 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] list_limit = None {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 15:45:07.986392 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 15:45:07.986449 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] log_config_append = None {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 15:45:07.986510 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 15:45:07.986577 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 15:45:07.986577 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] log_dir = None {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 15:45:07.986686 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 15:45:07.986743 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] log_file = None {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 15:45:07.986804 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 15:45:07.986872 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 15:45:07.986941 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 15:45:07.986941 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 15:45:07.987052 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 15:45:07.987116 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 15:45:07.987179 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 15:45:07.987244 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 15:45:07.987321 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 15:45:07.987321 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 15:45:07.987433 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 15:45:07.987489 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 15:45:07.987553 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 15:45:07.987619 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 15:45:07.987682 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 15:45:07.987740 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 15:45:07.987796 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 15:45:07.987859 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 15:45:07.987922 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 15:45:07.987989 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 15:45:07.988048 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 15:45:07.988109 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 15:45:07.988171 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 15:45:07.988243 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 15:45:07.988454 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 15:45:07.988539 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.20/identity {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 15:45:07.988713 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 15:45:07.988790 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] publish_errors = False {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 15:45:07.988960 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 15:45:07.989150 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 15:45:07.989306 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 15:45:07.989438 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 15:45:07.989558 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.20/identity {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 15:45:07.989671 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 15:45:07.989790 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] publish_errors = False {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 15:45:07.989901 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 15:45:07.990010 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 15:45:07.990126 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 15:45:07.990255 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] standard_threads = False {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 15:45:07.990377 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 15:45:07.990499 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 15:45:07.990617 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 15:45:07.990734 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 15:45:07.990854 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 15:45:07.990964 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 15:45:07.990964 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] use_journal = False {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 15:45:07.991171 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] standard_threads = False {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 15:45:07.991287 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] use_json = False {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 15:45:07.991397 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 15:45:07.991563 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] use_stderr = False {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 15:45:07.991563 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] use_syslog = False {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 15:45:07.991774 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 15:45:07.991774 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 15:45:07.991985 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 15:45:07.992095 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] use_journal = False {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 15:45:07.992214 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:07.992324 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] use_json = False {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 15:45:07.992324 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] use_stderr = False {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 15:45:07.992534 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:07.992629 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] use_syslog = False {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 15:45:07.992728 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:07.992837 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 15:45:07.992956 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:07.993068 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:07.993177 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:07.993304 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:07.993425 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:07.993549 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:07.993661 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:07.993770 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:07.993879 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:07.993989 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:07.994104 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:07.994222 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:07.994334 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:07.994450 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:07.994566 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:07.994678 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:07.994802 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:07.994925 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:07.995036 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:07.995150 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:07.995267 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:07.995267 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:07.995479 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:07.995597 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:07.995711 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:07.995835 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:07.995950 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:07.995950 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:07.996162 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:07.996280 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:07.996391 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:07.996391 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:07.996601 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:07.997050 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:07.997168 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:07.997372 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:07.997487 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:07.997596 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:07.997705 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:07.997815 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:07.997932 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:07.998043 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:07.998152 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:07.998260 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:07.998369 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:07.998477 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:07.998594 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:07.998705 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:07.998820 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:07.998930 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:07.999038 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:07.999146 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:07.999263 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:07.999379 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:07.999489 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:07.999597 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:07.999706 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:07.999814 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:07.999932 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.000042 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.000157 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.000267 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.000381 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.000491 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.000612 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.000722 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.000837 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.000954 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.001064 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.001172 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.001313 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.001433 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.001543 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.001652 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.001761 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.001869 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.001987 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.002097 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.002206 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.002314 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.002428 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.002544 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.002730 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.002844 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.002956 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.003066 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.003182 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.003298 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.003417 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.003538 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.003653 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.003763 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.003763 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.003974 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.003974 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.004192 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.004192 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.004404 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.004519 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.004628 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.004734 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.004828 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.004922 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.005015 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.005119 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.005238 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.005354 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.005450 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.005543 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.005670 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.005670 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.005864 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.005968 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.006077 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.006187 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.006297 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.006397 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.006513 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.006624 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.006741 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.006852 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.006968 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.007078 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.007197 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.007308 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.007431 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.007542 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.007653 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.007777 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.007888 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.007998 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.008115 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.008227 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.008345 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.008456 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.008565 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.008665 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.008781 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.008892 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.008999 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.009109 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.009290 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.009290 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.009517 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.009630 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.009740 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.009740 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.009925 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.010032 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.010150 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.010261 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.010371 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.010487 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.010599 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.010714 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.010833 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.010946 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.011055 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.011171 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.011282 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.011392 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.011509 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.011627 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.011738 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.011847 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.011956 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.012065 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.012186 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.012298 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.012408 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.012517 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.012626 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.012734 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.012852 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.012966 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.013075 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.013186 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.013312 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.013312 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.013603 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.013718 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.013834 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.013946 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.014055 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.014173 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.014296 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.014487 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.014604 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.014715 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.014825 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.014936 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.015046 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.015046 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.015266 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.015376 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.015486 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.015596 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.015706 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.015823 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.015935 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.016045 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.016154 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.016263 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.016373 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.016490 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.016602 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.016712 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.016836 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] auth.external = None {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.016948 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.017066 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.017209 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.017389 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.017513 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.017625 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.017737 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.017847 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] auth.password = None {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.017966 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] auth.external = None {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.018078 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] auth.token = None {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.018197 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.018311 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.018422 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.018533 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.018643 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.018755 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.018873 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] auth.password = None {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.018985 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.019096 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] auth.token = None {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.019206 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.019316 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.019426 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.019545 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.019657 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.019769 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.019879 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] credential.caching = True {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.019995 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.020107 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.020224 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.020337 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.020447 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.020558 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.020668 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.020786 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.020907 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] credential.caching = True {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.021019 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.021135 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.021261 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.021373 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.021483 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.021605 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.021719 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.021719 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.021926 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.021926 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.022140 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.022260 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.022372 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.022483 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.022592 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.022703 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.022813 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.022937 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.023050 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.023160 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.023269 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.023379 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.023489 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] federation.caching = True {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.023489 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.023705 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.023815 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.023925 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.024042 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.024153 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.024271 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.024383 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.024493 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.024604 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] federation.caching = True {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.024713 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.024834 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.024946 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.025058 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.025168 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.025304 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.025416 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.025538 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.025657 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.025807 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.025923 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.026041 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.026153 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] identity.caching = True {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.026272 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.026385 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.026494 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.026604 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.026723 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.026841 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.027027 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.027143 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.027254 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.027364 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.027474 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] identity.caching = True {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.027584 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.027702 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.027815 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.027926 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.028036 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.028148 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.028258 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.028377 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.028497 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.028612 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.028723 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.028833 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.028945 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.029064 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.029177 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.029177 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.029402 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.029513 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.029623 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.029745 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.029859 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.029970 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.030080 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.030190 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.030301 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.030419 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.030531 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.030641 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.030751 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.030861 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.030970 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.031089 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.031201 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.031313 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.031422 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.031533 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.031643 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.031761 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.031872 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.031983 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.032093 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.032203 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.032389 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.032515 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.032635 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.032747 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.032858 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.032972 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.033082 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.033200 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.033334 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.033446 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.033566 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.033680 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.033790 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.033908 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.034020 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.034138 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.034138 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.034345 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.034465 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.034577 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.034688 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.034797 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.034907 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.035017 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.035144 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.035257 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.035367 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.035479 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.035589 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.035698 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.035818 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.035931 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.036041 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.036152 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.036262 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.036372 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.036494 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.036607 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.036717 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.036827 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.036939 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.037064 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.037288 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.037406 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.037406 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.037615 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.037725 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.037836 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] ldap.user = None {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.037963 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.038077 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.038188 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.038298 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.038410 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.038521 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.038641 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.038753 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.038865 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.038975 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.039085 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.039195 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.039313 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] ldap.user = None {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.039426 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.039542 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.039654 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.039654 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.039867 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.039988 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.040100 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.040210 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.040320 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.040430 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.040540 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.040659 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.040772 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.040890 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.041002 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.041112 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.041240 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.041365 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.041479 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.041589 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.041699 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.041810 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.041919 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.042038 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.042150 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.042261 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.042371 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.042480 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.042591 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.042710 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.042822 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.042937 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.043047 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.043157 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.043267 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.043386 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.043498 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.043608 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.043718 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.043828 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.043939 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.044058 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.044170 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.044279 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.044389 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.044498 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.044608 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.044726 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.044838 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.044949 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.045058 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.045058 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.045377 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.045504 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.045651 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.045764 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.045875 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.045987 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.046097 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.046218 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.046331 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] resource.caching = True {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.046442 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.046553 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.046663 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.046774 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.046897 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.047011 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.047121 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.047232 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.047342 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.047452 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.047571 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.047684 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.047794 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] resource.caching = True {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.047905 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.048015 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.048126 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.048245 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.048363 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.048363 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] role.caching = True {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.048571 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.048681 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] role.driver = sql {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.048869 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.048985 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.049096 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.049210 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.049336 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.049447 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.049571 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.049685 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.049795 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.049906 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.050016 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.050016 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] role.caching = True {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.050231 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.050231 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.050440 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] role.driver = sql {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.050440 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.050645 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.050762 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.050882 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.050882 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.051089 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.051200 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.051310 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.051420 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.051538 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.051651 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.051761 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.051871 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.051981 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.051981 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.052195 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.052308 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.052418 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.052528 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.052638 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.052748 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.052867 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.052981 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.053093 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.053203 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.053364 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.053475 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.053598 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.053711 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.053822 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.053932 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.054041 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.054163 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.054316 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.054430 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.054542 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.054542 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.054749 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.054873 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.054986 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.055098 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.055268 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.055422 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.055533 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.055533 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.055751 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.055861 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.055970 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.056089 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.056202 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.056321 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.056433 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.056544 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.056661 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] token.caching = True {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.056772 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.056883 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.057001 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.057113 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.057247 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.057362 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.057472 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.057583 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.057706 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.057819 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.057932 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.058043 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.058154 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] token.caching = True {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.058265 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.058383 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.058495 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.058607 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.058716 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.058826 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.058936 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.059054 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.059166 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.059277 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.059388 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.059498 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.059609 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.059728 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.059840 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.059950 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.060060 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.060179 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.060299 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.060411 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.060534 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.060648 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.060760 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] database.connection = **** {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.060870 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.060982 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.061092 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.061253 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.061368 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.061480 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.061592 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.061717 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.061839 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.061952 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.061952 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.062167 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.062280 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.062390 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] database.connection = **** {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.062500 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.062610 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.062720 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.062840 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.062955 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.063066 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.063176 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.063286 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.063396 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.063515 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.063627 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.063737 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.063848 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.063958 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.064068 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.064188 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.064300 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.064410 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.064520 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.064630 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.064740 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.064860 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.064972 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 30 15:45:08.065084 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.065084 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.065084 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.065084 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.065512 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.065512 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.065728 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 15:45:08.065838 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81512) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 30 15:45:08.093142 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81511) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 15:45:08.093380 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81511) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 15:45:08.093595 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81511) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 15:45:08.094300 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81512) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 15:45:08.094535 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81512) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 15:45:08.094763 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81512) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 15:45:08.101834 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81511) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 15:45:08.103238 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81512) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 15:45:08.129864 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81511) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 15:45:08.130059 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81511) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 15:45:08.131631 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81512) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 15:45:08.131834 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81512) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 15:45:08.183059 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81511) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 15:45:08.184213 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81512) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 15:45:08.231030 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81511) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 15:45:08.231241 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81511) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 15:45:08.232786 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81512) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 15:45:08.232982 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81512) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 15:45:08.389794 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81511) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 15:45:08.390094 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81511) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 15:45:08.390219 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81511) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 15:45:08.390436 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81511) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 15:45:08.390632 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81511) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 15:45:08.390845 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81511) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 15:45:08.391546 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81512) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 15:45:08.391759 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81512) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 15:45:08.392005 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81512) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 15:45:08.392185 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81512) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 15:45:08.392429 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81512) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 15:45:08.392596 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81512) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 15:45:08.413758 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81511) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 15:45:08.413940 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81511) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 15:45:08.414192 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81511) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 15:45:08.414378 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81512) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 15:45:08.414505 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81511) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 15:45:08.414620 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81512) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 15:45:08.414732 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81511) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 15:45:08.414843 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81512) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 15:45:08.414963 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81511) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 15:45:08.415078 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81512) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 15:45:08.415188 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81511) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 15:45:08.415308 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81512) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 15:45:08.415308 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81512) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 15:45:08.415528 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81512) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 15:45:08.425930 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81511) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 15:45:08.426521 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81512) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 15:45:08.438860 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81511) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 15:45:08.439100 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81511) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 15:45:08.439236 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81512) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 15:45:08.439369 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81511) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 15:45:08.439486 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81512) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 15:45:08.439599 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81511) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 15:45:08.439708 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81512) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 15:45:08.439819 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81511) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 15:45:08.439939 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81512) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 15:45:08.440051 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81511) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 15:45:08.440162 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81512) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 15:45:08.440274 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81511) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 15:45:08.440384 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81512) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 15:45:08.440384 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81512) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 15:45:08.456723 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81511) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 15:45:08.456911 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81511) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 30 15:45:08.457262 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81511) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 15:45:08.457428 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81512) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 15:45:08.457552 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81511) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 30 15:45:08.457687 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81512) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 30 15:45:08.457803 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81511) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 15:45:08.457914 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81512) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 15:45:08.458025 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81511) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 15:45:08.458137 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81512) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 30 15:45:08.458263 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81511) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 15:45:08.458376 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81512) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 15:45:08.458487 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81511) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 15:45:08.458598 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81512) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 15:45:08.458785 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81512) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 15:45:08.459306 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81512) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 15:45:08.468044 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81511) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 15:45:08.468318 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81511) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 15:45:08.468586 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81511) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 15:45:08.468753 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81512) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 15:45:08.468876 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81511) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 15:45:08.469012 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81512) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 15:45:08.469191 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81512) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 15:45:08.469532 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81512) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 15:45:08.475918 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81511) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 15:45:08.476424 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81512) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 15:45:08.478785 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81511) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 15:45:08.479064 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81511) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 15:45:08.479317 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81512) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 15:45:08.479486 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81511) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 15:45:08.479609 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81511) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 15:45:08.479723 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81512) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 15:45:08.479856 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81511) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 15:45:08.479971 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81512) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 15:45:08.480084 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81511) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 15:45:08.480195 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81512) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 15:45:08.480336 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81511) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 15:45:08.480466 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81512) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 15:45:08.480606 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81511) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 15:45:08.480725 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81512) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 15:45:08.480874 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81511) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 15:45:08.480997 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81512) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 15:45:08.481118 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81511) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 15:45:08.481259 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81512) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 15:45:08.481413 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81511) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 15:45:08.481537 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81512) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 15:45:08.481824 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81512) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 15:45:08.482093 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81512) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 15:45:08.504311 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81511) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 15:45:08.504946 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81512) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 15:45:08.505893 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81511) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 15:45:08.506174 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81511) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 15:45:08.506432 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81512) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 15:45:08.506714 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81512) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 15:45:08.510621 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81511) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 15:45:08.510903 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81511) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 15:45:08.511175 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81511) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 15:45:08.511341 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81512) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 15:45:08.511622 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81512) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 15:45:08.511902 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81512) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 15:45:08.517579 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81511) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 15:45:08.517863 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81511) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 15:45:08.518339 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81512) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 15:45:08.518625 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81512) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 15:45:08.521853 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81511) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 15:45:08.522138 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81511) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 15:45:08.522417 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81511) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 15:45:08.522696 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81511) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 15:45:08.522891 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81512) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 15:45:08.523036 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81511) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 15:45:08.523161 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81512) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 15:45:08.523275 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81512) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 15:45:08.523549 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81512) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 15:45:08.523827 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81512) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 15:45:08.524102 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81512) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 15:45:08.524374 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81512) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 15:45:08.524643 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81512) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 15:45:08.524891 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81511) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 15:45:08.525170 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81511) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 15:45:08.525458 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81511) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 15:45:08.540776 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81511) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 15:45:08.541043 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81511) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 15:45:08.541630 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81512) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 15:45:08.541910 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81512) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 15:45:08.544850 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81511) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 15:45:08.546455 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81512) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 15:45:08.547956 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81511) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 15:45:08.548248 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81511) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 15:45:08.549963 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81512) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 15:45:08.550262 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81512) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 15:45:08.552594 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81511) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 15:45:08.554462 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81512) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 15:45:08.555688 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81511) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 15:45:08.557595 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81512) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 15:45:08.558738 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81511) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 15:45:08.559034 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81511) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 15:45:08.559317 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81511) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 15:45:08.559596 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81511) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 15:45:08.559882 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81511) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 15:45:08.560260 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81511) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 15:45:08.560773 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81512) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 15:45:08.561086 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81512) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 15:45:08.561452 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81512) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 15:45:08.561741 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81512) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 15:45:08.562036 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81512) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 15:45:08.562300 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81512) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 15:45:08.575961 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81511) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 15:45:08.576245 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81511) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 15:45:08.576509 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81511) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 15:45:08.576791 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81511) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 15:45:08.578078 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81512) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 15:45:08.578366 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81512) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 15:45:08.578636 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81512) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 15:45:08.578903 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81512) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 15:45:08.583496 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81511) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 15:45:08.585140 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81511) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 15:45:08.585796 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81512) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 15:45:08.586686 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81511) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 15:45:08.586958 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81511) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 15:45:08.587311 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81512) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 15:45:08.588845 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81512) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 15:45:08.589132 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81512) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 15:45:08.589588 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81511) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 15:45:08.589865 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81511) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 15:45:08.590164 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81511) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 15:45:08.590428 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81511) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 15:45:08.590713 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81511) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 15:45:08.591787 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81512) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 15:45:08.592072 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81512) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 15:45:08.592504 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81512) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 15:45:08.592798 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81512) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 15:45:08.593093 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81512) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 15:45:08.601944 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81511) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 15:45:08.602218 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81511) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 15:45:08.602488 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81511) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 15:45:08.602765 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81511) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 15:45:08.603092 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81511) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 15:45:08.603290 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81511) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 15:45:08.603592 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81511) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 15:45:08.604183 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81512) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 15:45:08.604464 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81512) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 15:45:08.604843 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81512) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 15:45:08.605122 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81512) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 15:45:08.605408 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81512) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 15:45:08.605715 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81512) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 15:45:08.606014 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81512) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 15:45:08.618858 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81511) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 15:45:08.621318 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81512) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 15:45:08.622199 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81511) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 15:45:08.624602 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81512) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 15:45:08.625059 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81511) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 15:45:08.630316 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81511) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 15:45:08.630529 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81512) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 15:45:08.630660 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81511) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 15:45:08.630660 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81511) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 15:45:08.630660 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81511) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 15:45:08.630982 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81512) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 15:45:08.630982 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81512) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 15:45:08.630982 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81512) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 15:45:08.631304 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81512) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 15:45:08.635036 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81511) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 15:45:08.636493 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81511) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 15:45:08.637524 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81512) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 15:45:08.639002 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81512) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 15:45:08.639409 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81511) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 15:45:08.643970 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81511) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 15:45:08.643970 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81511) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 15:45:08.643970 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81511) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 15:45:08.645009 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81512) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 15:45:08.645009 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81512) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 15:45:08.645009 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81512) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 15:45:08.645009 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81512) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 15:45:08.648166 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81511) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 15:45:08.648452 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81511) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 15:45:08.648724 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81511) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 15:45:08.650705 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81512) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 15:45:08.650997 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81512) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 15:45:08.651278 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81512) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 15:45:08.655574 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81511) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 15:45:08.655852 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81511) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 15:45:08.656129 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81511) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 15:45:08.656422 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81511) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 15:45:08.656683 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81511) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 15:45:08.656955 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81511) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 15:45:08.657259 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81511) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 15:45:08.657523 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81511) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 15:45:08.657803 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81511) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 15:45:08.658073 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81511) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 15:45:08.658259 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81512) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 15:45:08.658388 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81511) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 15:45:08.658514 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81512) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 15:45:08.658637 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81511) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 15:45:08.658765 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81512) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 15:45:08.658879 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81511) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 15:45:08.658990 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81512) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 15:45:08.659102 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81511) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 15:45:08.659215 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81512) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 15:45:08.659519 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81512) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 15:45:08.659779 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81512) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 15:45:08.660060 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81512) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 15:45:08.660341 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81512) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 15:45:08.660614 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81512) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 15:45:08.660889 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81512) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 15:45:08.661149 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81512) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 15:45:08.661513 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81512) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 15:45:08.661793 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81512) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 15:45:08.692837 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55adf673e9c0 pid: 81511 (default app) Aug 30 15:45:08.696423 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3c156a-2456-403e-a7d4-4bad3054e6b4 None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:08.696708 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3c156a-2456-403e-a7d4-4bad3054e6b4 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:08.696977 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3c156a-2456-403e-a7d4-4bad3054e6b4 None None] PATH_INFO: `/v3` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:08.701230 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1/1] 10.222.0.20 () {50 vars in 791 bytes} [Tue Aug 30 15:45:08 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:45:08.702890 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55adf673e9c0 pid: 81512 (default app) Aug 30 15:45:15.278942 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5f9d70-3e30-4560-a067-752e468d98ef None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:15.279502 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5f9d70-3e30-4560-a067-752e468d98ef None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:15.279502 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5f9d70-3e30-4560-a067-752e468d98ef None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:15.280902 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 2/2] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 30 15:45:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 15:45:15.291023 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2df5172-9500-49ab-8f21-bdf2c0565c52 None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:15.291815 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2df5172-9500-49ab-8f21-bdf2c0565c52 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:15.291815 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2df5172-9500-49ab-8f21-bdf2c0565c52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:15.368805 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: INFO dbcounter [None req-b2df5172-9500-49ab-8f21-bdf2c0565c52 None None] Registered counter for database keystone Aug 30 15:45:15.390392 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG oslo_db.sqlalchemy.engines [None req-b2df5172-9500-49ab-8f21-bdf2c0565c52 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81512) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 30 15:45:15.401833 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writer thread running {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 30 15:45:15.589888 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG passlib.handlers.bcrypt [None req-b2df5172-9500-49ab-8f21-bdf2c0565c52 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81512) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 30 15:45:15.590186 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG passlib.handlers.bcrypt [None req-b2df5172-9500-49ab-8f21-bdf2c0565c52 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81512) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 30 15:45:15.649087 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-b2df5172-9500-49ab-8f21-bdf2c0565c52 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:45:15.650727 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-b2df5172-9500-49ab-8f21-bdf2c0565c52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:45:15.803369 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1/3] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 30 15:45:15 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 518 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 30 15:45:16.302385 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5f9d70-3e30-4560-a067-752e468d98ef None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:45:16.381602 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: INFO dbcounter [None req-3b5f9d70-3e30-4560-a067-752e468d98ef None None] Registered counter for database keystone Aug 30 15:45:16.406778 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG oslo_db.sqlalchemy.engines [None req-3b5f9d70-3e30-4560-a067-752e468d98ef None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81511) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 30 15:45:16.418243 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writer thread running {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 30 15:45:16.538864 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5f9d70-3e30-4560-a067-752e468d98ef None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:45:16.544383 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d521d9-a7c2-4168-9f7e-c1d61885173b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:45:16.545016 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d521d9-a7c2-4168-9f7e-c1d61885173b admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:16.545320 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d521d9-a7c2-4168-9f7e-c1d61885173b admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:16.545590 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d521d9-a7c2-4168-9f7e-c1d61885173b admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:16.554776 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d521d9-a7c2-4168-9f7e-c1d61885173b admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:45:16.605253 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: WARNING keystone.common.rbac_enforcer.enforcer [None req-09d521d9-a7c2-4168-9f7e-c1d61885173b admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 30 15:45:16.605620 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d521d9-a7c2-4168-9f7e-c1d61885173b admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:45:16.615709 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 3/4] 10.222.0.20 () {58 vars in 1324 bytes} [Tue Aug 30 15:45:16 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 30 15:45:16.623284 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2df5172-9500-49ab-8f21-bdf2c0565c52 None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:45:16.639785 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2df5172-9500-49ab-8f21-bdf2c0565c52 None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:45:16.645674 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db5117e-f5a9-44c3-b37a-f3b2f34ccdf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:45:16.647089 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db5117e-f5a9-44c3-b37a-f3b2f34ccdf7 admin admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:16.647368 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db5117e-f5a9-44c3-b37a-f3b2f34ccdf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:16.647636 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db5117e-f5a9-44c3-b37a-f3b2f34ccdf7 admin admin] PATH_INFO: `/v3/projects/f6954fa5b9a94855887d7068fc788d39` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:16.649334 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db5117e-f5a9-44c3-b37a-f3b2f34ccdf7 admin admin] RBAC: Authorizing `identity:get_project(project_id=f6954fa5b9a94855887d7068fc788d39)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:45:16.701108 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: WARNING keystone.common.rbac_enforcer.enforcer [None req-2db5117e-f5a9-44c3-b37a-f3b2f34ccdf7 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 30 15:45:16.701472 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db5117e-f5a9-44c3-b37a-f3b2f34ccdf7 admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:45:16.706086 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 2/5] 10.222.0.20 () {56 vars in 1207 bytes} [Tue Aug 30 15:45:16 2022] GET /identity/v3/projects/f6954fa5b9a94855887d7068fc788d39 => generated 344 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:45:18.563637 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b23e10-3e22-40f1-b6e9-b35a620fd093 None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:18.564528 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b23e10-3e22-40f1-b6e9-b35a620fd093 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:18.564528 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b23e10-3e22-40f1-b6e9-b35a620fd093 None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:18.571414 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 4/6] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 30 15:45:18 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 15:45:18.576941 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0efc50-3033-4b0c-9ead-1c34e35ff0fd None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:18.577390 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0efc50-3033-4b0c-9ead-1c34e35ff0fd None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:18.577490 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0efc50-3033-4b0c-9ead-1c34e35ff0fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:18.599676 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-7c0efc50-3033-4b0c-9ead-1c34e35ff0fd None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:45:18.601169 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-7c0efc50-3033-4b0c-9ead-1c34e35ff0fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:45:18.610158 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 3/7] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 30 15:45:18 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 30 15:45:18.874881 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b23e10-3e22-40f1-b6e9-b35a620fd093 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:45:18.890384 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b23e10-3e22-40f1-b6e9-b35a620fd093 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:45:18.895878 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c5bb0d-9dd8-446a-a0bf-2072c2aa3cc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:45:18.896492 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c5bb0d-9dd8-446a-a0bf-2072c2aa3cc0 admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:18.896771 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c5bb0d-9dd8-446a-a0bf-2072c2aa3cc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:18.897040 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c5bb0d-9dd8-446a-a0bf-2072c2aa3cc0 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:18.906469 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c5bb0d-9dd8-446a-a0bf-2072c2aa3cc0 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:45:18.908279 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c5bb0d-9dd8-446a-a0bf-2072c2aa3cc0 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:45:18.917655 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 5/8] 10.222.0.20 () {58 vars in 1324 bytes} [Tue Aug 30 15:45:18 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 30 15:45:18.926026 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0efc50-3033-4b0c-9ead-1c34e35ff0fd None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:45:18.939540 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0efc50-3033-4b0c-9ead-1c34e35ff0fd None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:45:18.945312 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34e93c3-bb82-4324-8293-f0395a163a6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:45:18.945939 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34e93c3-bb82-4324-8293-f0395a163a6d admin admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:18.946216 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34e93c3-bb82-4324-8293-f0395a163a6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:18.946481 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34e93c3-bb82-4324-8293-f0395a163a6d admin admin] PATH_INFO: `/v3/users/8b9160d3cbae47a48dc9dee882da999b` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:18.947928 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34e93c3-bb82-4324-8293-f0395a163a6d admin admin] RBAC: Authorizing `identity:get_user(user_id=8b9160d3cbae47a48dc9dee882da999b)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:45:18.950139 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34e93c3-bb82-4324-8293-f0395a163a6d admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:45:18.951887 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 4/9] 10.222.0.20 () {56 vars in 1198 bytes} [Tue Aug 30 15:45:18 2022] GET /identity/v3/users/8b9160d3cbae47a48dc9dee882da999b => generated 247 bytes in 27 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:45:20.914771 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac07e336-bc28-45fb-ae2c-3790e55bd1c5 None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:20.915285 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac07e336-bc28-45fb-ae2c-3790e55bd1c5 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:20.915285 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac07e336-bc28-45fb-ae2c-3790e55bd1c5 None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:20.916561 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 6/10] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 30 15:45:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 15:45:20.921935 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164e4d04-e286-4cf0-9440-169d20d873ac None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:20.922358 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164e4d04-e286-4cf0-9440-169d20d873ac None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:20.922532 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164e4d04-e286-4cf0-9440-169d20d873ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:20.941087 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-164e4d04-e286-4cf0-9440-169d20d873ac None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:45:20.942322 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-164e4d04-e286-4cf0-9440-169d20d873ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:45:20.944381 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6694d9-8c8a-4824-8a89-15a61996e41f None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:20.944666 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6694d9-8c8a-4824-8a89-15a61996e41f None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:20.944938 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6694d9-8c8a-4824-8a89-15a61996e41f None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:20.946323 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 7/11] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 30 15:45:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 15:45:20.951372 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4aea643-a0f2-44d5-9d7f-6987038e4d1e None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:20.951677 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4aea643-a0f2-44d5-9d7f-6987038e4d1e None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:20.951958 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4aea643-a0f2-44d5-9d7f-6987038e4d1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:20.994539 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG passlib.handlers.bcrypt [None req-a4aea643-a0f2-44d5-9d7f-6987038e4d1e None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81511) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 30 15:45:20.994936 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG passlib.handlers.bcrypt [None req-a4aea643-a0f2-44d5-9d7f-6987038e4d1e None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81511) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 30 15:45:21.024701 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 5/12] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 30 15:45:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 15:45:21.043937 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-a4aea643-a0f2-44d5-9d7f-6987038e4d1e None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:45:21.045571 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-a4aea643-a0f2-44d5-9d7f-6987038e4d1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:45:21.147173 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 8/13] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 30 15:45:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 15:45:21.284578 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164e4d04-e286-4cf0-9440-169d20d873ac None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:45:21.310376 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:45:21.401934 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164e4d04-e286-4cf0-9440-169d20d873ac None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:45:21.406420 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4aea643-a0f2-44d5-9d7f-6987038e4d1e None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:45:21.438450 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5f9a75-5dcd-46d3-b340-451940defc7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:45:21.439098 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5f9a75-5dcd-46d3-b340-451940defc7d None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:21.439393 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5f9a75-5dcd-46d3-b340-451940defc7d None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:21.439644 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5f9a75-5dcd-46d3-b340-451940defc7d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:21.450588 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5f9a75-5dcd-46d3-b340-451940defc7d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 15:45:21.459734 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application [None req-6d5f9a75-5dcd-46d3-b340-451940defc7d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 15:45:21.459734 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:45:21.459734 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:45:21.459734 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:45:21.459734 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:45:21.459734 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:45:21.459734 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:45:21.459734 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:45:21.459734 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:45:21.459734 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:45:21.459734 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:45:21.459734 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:45:21.459734 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 15:45:21.459734 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 15:45:21.459734 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 15:45:21.459734 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 15:45:21.460718 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 15:45:21.460718 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 15:45:21.460718 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 15:45:21.460718 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 15:45:21.460718 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:45:21.460718 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:45:21.460718 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:45:21.460718 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:45:21.460718 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:45:21.460718 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:45:21.460718 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:45:21.460718 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application with Lock( Aug 30 15:45:21.460718 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:45:21.460718 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:45:21.460718 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:45:21.460718 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:45:21.461621 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:45:21.461621 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:45:21.461621 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:45:21.461621 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:45:21.461621 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 15:45:21.461621 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 15:45:21.461621 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 15:45:21.461621 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 15:45:21.461621 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 15:45:21.461621 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 15:45:21.461621 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 15:45:21.461621 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Aug 30 15:45:21.461621 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 6/14] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Aug 30 15:45:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 15:45:21.466912 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5f9a75-5dcd-46d3-b340-451940defc7d None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:45:21.509021 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4aea643-a0f2-44d5-9d7f-6987038e4d1e None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:45:21.511335 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:45:21.546280 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7eb1578-41d3-4d80-a793-cdb6c53d21d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:45:21.547132 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7eb1578-41d3-4d80-a793-cdb6c53d21d8 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:21.547414 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7eb1578-41d3-4d80-a793-cdb6c53d21d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:21.547681 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7eb1578-41d3-4d80-a793-cdb6c53d21d8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:21.549516 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:45:21.568560 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5f9a75-5dcd-46d3-b340-451940defc7d None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:45:21.607356 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47e4b78-b997-4e95-9d74-6f3d84d76681 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:45:21.608215 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47e4b78-b997-4e95-9d74-6f3d84d76681 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:21.608498 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47e4b78-b997-4e95-9d74-6f3d84d76681 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:21.608771 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47e4b78-b997-4e95-9d74-6f3d84d76681 None admin] PATH_INFO: `/v3/roles` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:21.609729 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47e4b78-b997-4e95-9d74-6f3d84d76681 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81512) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:45:21.610032 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47e4b78-b997-4e95-9d74-6f3d84d76681 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:45:21.611486 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47e4b78-b997-4e95-9d74-6f3d84d76681 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:45:21.623110 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7eb1578-41d3-4d80-a793-cdb6c53d21d8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:45:21.624325 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 7/15] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Aug 30 15:45:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 15:45:21.624636 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7eb1578-41d3-4d80-a793-cdb6c53d21d8 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:45:21.630285 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47e4b78-b997-4e95-9d74-6f3d84d76681 None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:45:21.725495 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 9/16] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Aug 30 15:45:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 30 15:45:21.727765 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47e4b78-b997-4e95-9d74-6f3d84d76681 None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:45:21.731083 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7eb1578-41d3-4d80-a793-cdb6c53d21d8 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:45:21.733873 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:45:21.763151 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdce90f0-bd61-45c5-84ad-b25c1739efaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:45:21.763780 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdce90f0-bd61-45c5-84ad-b25c1739efaf None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:21.764054 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdce90f0-bd61-45c5-84ad-b25c1739efaf None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:21.764315 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdce90f0-bd61-45c5-84ad-b25c1739efaf None admin] PATH_INFO: `/v3/users/8b9160d3cbae47a48dc9dee882da999b` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:21.765488 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdce90f0-bd61-45c5-84ad-b25c1739efaf None admin] RBAC: Authorizing `identity:get_user(user_id=8b9160d3cbae47a48dc9dee882da999b)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:45:21.766979 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdce90f0-bd61-45c5-84ad-b25c1739efaf None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:45:21.768544 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 8/17] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Aug 30 15:45:21 2022] GET /identity/v3/users/8b9160d3cbae47a48dc9dee882da999b => generated 247 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:45:21.774624 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdce90f0-bd61-45c5-84ad-b25c1739efaf None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:45:21.789385 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:45:21.828525 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7eb1578-41d3-4d80-a793-cdb6c53d21d8 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:45:21.860916 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c67d88-fad8-4911-a544-7d5d8dc15dbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:45:21.861828 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c67d88-fad8-4911-a544-7d5d8dc15dbd None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:21.862109 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c67d88-fad8-4911-a544-7d5d8dc15dbd None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:21.862367 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c67d88-fad8-4911-a544-7d5d8dc15dbd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:21.874090 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdce90f0-bd61-45c5-84ad-b25c1739efaf None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:45:21.883589 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c67d88-fad8-4911-a544-7d5d8dc15dbd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:45:21.885089 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c67d88-fad8-4911-a544-7d5d8dc15dbd None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:45:21.905071 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-41c67d88-fad8-4911-a544-7d5d8dc15dbd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 15:45:21.905071 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:45:21.905071 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:45:21.905071 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:45:21.905071 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:45:21.905071 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:45:21.905071 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:45:21.905071 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:45:21.905071 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:45:21.905071 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:45:21.905071 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:45:21.905071 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:45:21.905071 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 15:45:21.905071 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 15:45:21.905071 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 15:45:21.905071 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 15:45:21.906282 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:45:21.906282 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:45:21.906282 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:45:21.906282 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:45:21.906282 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:45:21.906282 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:45:21.906282 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:45:21.906282 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Aug 30 15:45:21.906282 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:45:21.906282 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:45:21.906282 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:45:21.906282 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:45:21.906282 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:45:21.906282 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:45:21.906282 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:45:21.906282 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:45:21.906282 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 15:45:21.907227 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 15:45:21.907227 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 15:45:21.907227 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Aug 30 15:45:21.907227 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 10/18] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 30 15:45:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 15:45:21.911313 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0be3b0-af58-40b6-9f13-bfa58d052d44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:45:21.911958 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0be3b0-af58-40b6-9f13-bfa58d052d44 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:21.912091 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c67d88-fad8-4911-a544-7d5d8dc15dbd None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:45:21.912251 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0be3b0-af58-40b6-9f13-bfa58d052d44 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:21.912495 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0be3b0-af58-40b6-9f13-bfa58d052d44 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:21.913762 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0be3b0-af58-40b6-9f13-bfa58d052d44 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:45:21.915241 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0be3b0-af58-40b6-9f13-bfa58d052d44 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:45:21.917186 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 9/19] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 30 15:45:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 15:45:21.922903 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0be3b0-af58-40b6-9f13-bfa58d052d44 None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:45:21.969449 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:45:22.003478 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:45:22.016884 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c67d88-fad8-4911-a544-7d5d8dc15dbd None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:45:22.023068 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0be3b0-af58-40b6-9f13-bfa58d052d44 None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:45:22.050121 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e4360d-b0c2-44be-98f8-8f592df87de5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:45:22.050701 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e4360d-b0c2-44be-98f8-8f592df87de5 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:22.050979 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e4360d-b0c2-44be-98f8-8f592df87de5 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:22.051244 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e4360d-b0c2-44be-98f8-8f592df87de5 None admin] PATH_INFO: `/v3/domains` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:22.051884 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e4360d-b0c2-44be-98f8-8f592df87de5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:45:22.052184 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e4360d-b0c2-44be-98f8-8f592df87de5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:45:22.053597 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e4360d-b0c2-44be-98f8-8f592df87de5 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:45:22.056043 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5736313-2ba8-4d6e-bc98-b603e1447613 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:45:22.056649 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5736313-2ba8-4d6e-bc98-b603e1447613 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:22.056930 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5736313-2ba8-4d6e-bc98-b603e1447613 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:22.057194 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5736313-2ba8-4d6e-bc98-b603e1447613 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:22.057953 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5736313-2ba8-4d6e-bc98-b603e1447613 None admin] RBAC: Adding query filter params (role.id=f265386950a641ac8413b53eda4ae34d, scope.domain.id=default, user.id=8b9160d3cbae47a48dc9dee882da999b) {{(pid=81512) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:45:22.058258 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5736313-2ba8-4d6e-bc98-b603e1447613 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:45:22.059679 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5736313-2ba8-4d6e-bc98-b603e1447613 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:45:22.066511 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 10/20] 10.222.0.20 () {56 vars in 1322 bytes} [Tue Aug 30 15:45:21 2022] GET /identity/v3/role_assignments?user.id=8b9160d3cbae47a48dc9dee882da999b&scope.domain.id=default&role.id=f265386950a641ac8413b53eda4ae34d => generated 233 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:45:22.071115 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 11/21] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Aug 30 15:45:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:45:24.216761 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce11d70-37f9-4d6c-9849-7840832f4282 None None] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:24.217307 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce11d70-37f9-4d6c-9849-7840832f4282 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:24.217307 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce11d70-37f9-4d6c-9849-7840832f4282 None None] PATH_INFO: `/` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:24.218940 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 11/22] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 30 15:45:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 15:45:24.224323 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0547a1-3db7-49e0-94db-ff06623cb0a2 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:24.224737 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0547a1-3db7-49e0-94db-ff06623cb0a2 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:24.224901 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0547a1-3db7-49e0-94db-ff06623cb0a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:24.254187 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-7f0547a1-3db7-49e0-94db-ff06623cb0a2 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:45:24.255403 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-7f0547a1-3db7-49e0-94db-ff06623cb0a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:45:24.333279 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 12/23] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 30 15:45:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 15:45:24.600500 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0450067f-7e0b-408b-bc32-facf8cecebdd None None] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:24.600764 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0450067f-7e0b-408b-bc32-facf8cecebdd None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:24.601027 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0450067f-7e0b-408b-bc32-facf8cecebdd None None] PATH_INFO: `/` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:24.602313 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 12/24] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 30 15:45:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 15:45:24.607736 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1159298-3021-4dcd-b1cc-46b7a5b48c53 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:24.608025 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1159298-3021-4dcd-b1cc-46b7a5b48c53 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:24.608298 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1159298-3021-4dcd-b1cc-46b7a5b48c53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:24.636601 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-a1159298-3021-4dcd-b1cc-46b7a5b48c53 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:45:24.637825 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-a1159298-3021-4dcd-b1cc-46b7a5b48c53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:45:24.651270 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0450067f-7e0b-408b-bc32-facf8cecebdd None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:45:24.721421 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:45:24.771826 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 13/25] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 30 15:45:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 15:45:24.793406 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6c72d8-b255-43bc-a0a8-dd9349d4007b None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:24.793703 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6c72d8-b255-43bc-a0a8-dd9349d4007b None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:24.793954 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6c72d8-b255-43bc-a0a8-dd9349d4007b None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:24.795554 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 14/26] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 30 15:45:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 15:45:24.800488 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f046cc8-8226-46d4-a312-3f8cc51f7c00 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:24.800767 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f046cc8-8226-46d4-a312-3f8cc51f7c00 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:24.801040 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f046cc8-8226-46d4-a312-3f8cc51f7c00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:24.816078 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0450067f-7e0b-408b-bc32-facf8cecebdd None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:45:24.828587 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-2f046cc8-8226-46d4-a312-3f8cc51f7c00 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:45:24.829835 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-2f046cc8-8226-46d4-a312-3f8cc51f7c00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:45:24.879051 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ecf80b-aed7-4fa2-bc93-9020307f7798 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:45:24.879607 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ecf80b-aed7-4fa2-bc93-9020307f7798 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:24.879885 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ecf80b-aed7-4fa2-bc93-9020307f7798 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:24.880155 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ecf80b-aed7-4fa2-bc93-9020307f7798 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:24.897410 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ecf80b-aed7-4fa2-bc93-9020307f7798 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 15:45:24.902746 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application [None req-40ecf80b-aed7-4fa2-bc93-9020307f7798 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 15:45:24.902746 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:45:24.902746 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:45:24.902746 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:45:24.902746 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:45:24.902746 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:45:24.902746 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:45:24.902746 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:45:24.902746 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:45:24.902746 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:45:24.902746 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:45:24.902746 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:45:24.902746 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 15:45:24.902746 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 15:45:24.902746 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 15:45:24.902746 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 15:45:24.903627 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 15:45:24.903627 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 15:45:24.903627 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 15:45:24.903627 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 15:45:24.903627 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:45:24.903627 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:45:24.903627 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:45:24.903627 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:45:24.903627 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:45:24.903627 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:45:24.903627 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:45:24.903627 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application with Lock( Aug 30 15:45:24.903627 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:45:24.903627 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:45:24.903627 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:45:24.903627 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:45:24.904395 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:45:24.904395 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:45:24.904395 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:45:24.904395 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:45:24.904395 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 15:45:24.904395 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 15:45:24.904395 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 15:45:24.904395 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 15:45:24.904395 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 15:45:24.904395 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 15:45:24.904395 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 15:45:24.904395 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Aug 30 15:45:24.904395 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 13/27] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Aug 30 15:45:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 15:45:24.911442 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ecf80b-aed7-4fa2-bc93-9020307f7798 None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:45:24.917921 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:45:24.953712 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 15/28] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 30 15:45:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 15:45:25.030388 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95313cc9-7f1e-4515-bd11-15f4596162ef None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:25.031173 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95313cc9-7f1e-4515-bd11-15f4596162ef None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:25.031754 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95313cc9-7f1e-4515-bd11-15f4596162ef None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:25.033378 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 16/29] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 30 15:45:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 15:45:25.040453 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115851c5-0a1c-4fe6-aa7a-1c5700a8effe None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:25.041273 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115851c5-0a1c-4fe6-aa7a-1c5700a8effe None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:25.042098 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115851c5-0a1c-4fe6-aa7a-1c5700a8effe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:25.055512 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ecf80b-aed7-4fa2-bc93-9020307f7798 None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:45:25.065527 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-115851c5-0a1c-4fe6-aa7a-1c5700a8effe None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:45:25.067156 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-115851c5-0a1c-4fe6-aa7a-1c5700a8effe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:45:25.130764 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66d3ef9-2168-4139-b108-24f0236ee30b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:45:25.131312 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66d3ef9-2168-4139-b108-24f0236ee30b None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:25.131587 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66d3ef9-2168-4139-b108-24f0236ee30b None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:25.131857 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66d3ef9-2168-4139-b108-24f0236ee30b None admin] PATH_INFO: `/v3/roles` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:25.132538 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66d3ef9-2168-4139-b108-24f0236ee30b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81512) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:45:25.132837 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66d3ef9-2168-4139-b108-24f0236ee30b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:45:25.134273 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66d3ef9-2168-4139-b108-24f0236ee30b None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:45:25.151066 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 14/30] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Aug 30 15:45:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:45:25.153199 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 17/31] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 30 15:45:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 15:45:25.158252 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66d3ef9-2168-4139-b108-24f0236ee30b None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:45:25.243503 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:45:25.286912 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66d3ef9-2168-4139-b108-24f0236ee30b None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:45:25.299932 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115851c5-0a1c-4fe6-aa7a-1c5700a8effe None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:45:25.351045 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:45:25.361940 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b26da1-9fff-4363-a49f-fac8eb492d12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:45:25.362597 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b26da1-9fff-4363-a49f-fac8eb492d12 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:25.362878 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b26da1-9fff-4363-a49f-fac8eb492d12 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:25.363155 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b26da1-9fff-4363-a49f-fac8eb492d12 None admin] PATH_INFO: `/v3/users/8b9160d3cbae47a48dc9dee882da999b` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:25.369921 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b26da1-9fff-4363-a49f-fac8eb492d12 None admin] RBAC: Authorizing `identity:get_user(user_id=8b9160d3cbae47a48dc9dee882da999b)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:45:25.371412 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b26da1-9fff-4363-a49f-fac8eb492d12 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:45:25.373667 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 15/32] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Aug 30 15:45:25 2022] GET /identity/v3/users/8b9160d3cbae47a48dc9dee882da999b => generated 247 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:45:25.374279 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b26da1-9fff-4363-a49f-fac8eb492d12 None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:45:25.437955 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115851c5-0a1c-4fe6-aa7a-1c5700a8effe None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:45:25.502263 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b220bb-a025-4f43-acfe-9afcbadfdd0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:45:25.503240 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b220bb-a025-4f43-acfe-9afcbadfdd0f None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:25.504952 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b220bb-a025-4f43-acfe-9afcbadfdd0f None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:25.505413 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b220bb-a025-4f43-acfe-9afcbadfdd0f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:25.524628 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b220bb-a025-4f43-acfe-9afcbadfdd0f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:45:25.526509 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b220bb-a025-4f43-acfe-9afcbadfdd0f None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:45:25.541573 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b26da1-9fff-4363-a49f-fac8eb492d12 None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:45:25.545195 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-f9b220bb-a025-4f43-acfe-9afcbadfdd0f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 15:45:25.545195 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:45:25.545195 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:45:25.545195 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:45:25.545195 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:45:25.545195 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:45:25.545195 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:45:25.545195 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:45:25.545195 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:45:25.545195 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:45:25.545195 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:45:25.545195 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:45:25.545195 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 15:45:25.545195 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 15:45:25.545195 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 15:45:25.545195 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 15:45:25.546061 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:45:25.546061 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:45:25.546061 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:45:25.546061 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:45:25.546061 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:45:25.546061 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:45:25.546061 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:45:25.546061 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Aug 30 15:45:25.546061 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:45:25.546061 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:45:25.546061 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:45:25.546061 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:45:25.546061 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:45:25.546061 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:45:25.546061 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:45:25.546061 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:45:25.546061 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 15:45:25.546887 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 15:45:25.546887 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 15:45:25.546887 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Aug 30 15:45:25.548623 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 18/33] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 30 15:45:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 15:45:25.550085 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b220bb-a025-4f43-acfe-9afcbadfdd0f None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:45:25.581668 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af17c9d-7a82-44f6-b40e-90320268bc30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:45:25.582744 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af17c9d-7a82-44f6-b40e-90320268bc30 None admin] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:25.583274 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af17c9d-7a82-44f6-b40e-90320268bc30 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:25.583798 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af17c9d-7a82-44f6-b40e-90320268bc30 None admin] PATH_INFO: `/v3/roles` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:25.584659 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af17c9d-7a82-44f6-b40e-90320268bc30 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:45:25.586540 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af17c9d-7a82-44f6-b40e-90320268bc30 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:45:25.590746 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:45:25.662115 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:45:25.675614 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b220bb-a025-4f43-acfe-9afcbadfdd0f None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:45:25.710088 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228bf68b-63dc-4bf5-b2fd-d7cc666a4ab8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:45:25.710640 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228bf68b-63dc-4bf5-b2fd-d7cc666a4ab8 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:25.710918 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228bf68b-63dc-4bf5-b2fd-d7cc666a4ab8 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:25.711188 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228bf68b-63dc-4bf5-b2fd-d7cc666a4ab8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:25.712553 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228bf68b-63dc-4bf5-b2fd-d7cc666a4ab8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:45:25.714059 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228bf68b-63dc-4bf5-b2fd-d7cc666a4ab8 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:45:25.715836 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 19/34] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 30 15:45:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:45:25.716372 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228bf68b-63dc-4bf5-b2fd-d7cc666a4ab8 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:45:25.726752 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 16/35] 10.222.0.20 () {60 vars in 1131 bytes} [Tue Aug 30 15:45:25 2022] POST /identity/v3/roles => generated 219 bytes in 353 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 15:45:25.728029 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af17c9d-7a82-44f6-b40e-90320268bc30 None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:45:25.831083 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228bf68b-63dc-4bf5-b2fd-d7cc666a4ab8 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:45:25.860918 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af17c9d-7a82-44f6-b40e-90320268bc30 None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:45:25.873244 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f740be-0d0b-41fd-9219-d61640cf790e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:45:25.874607 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f740be-0d0b-41fd-9219-d61640cf790e None admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:25.875385 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f740be-0d0b-41fd-9219-d61640cf790e None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:25.875918 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f740be-0d0b-41fd-9219-d61640cf790e None admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:25.876779 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f740be-0d0b-41fd-9219-d61640cf790e None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:45:25.878857 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f740be-0d0b-41fd-9219-d61640cf790e None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:45:25.905471 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af822c5-a87a-41bc-bb97-5be93223dd05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:45:25.906901 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af822c5-a87a-41bc-bb97-5be93223dd05 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:25.907185 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af822c5-a87a-41bc-bb97-5be93223dd05 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:25.908354 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af822c5-a87a-41bc-bb97-5be93223dd05 None admin] PATH_INFO: `/v3/domains` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:25.908810 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af822c5-a87a-41bc-bb97-5be93223dd05 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81512) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:45:25.909121 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af822c5-a87a-41bc-bb97-5be93223dd05 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:45:25.910785 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af822c5-a87a-41bc-bb97-5be93223dd05 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:45:25.929856 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 17/36] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Aug 30 15:45:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:45:25.931228 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c45602e-4627-402d-b66b-152f7374996a None None] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:25.931515 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c45602e-4627-402d-b66b-152f7374996a None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:25.931790 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c45602e-4627-402d-b66b-152f7374996a None None] PATH_INFO: `/` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:25.932993 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 18/37] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 30 15:45:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 15:45:25.933705 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c45602e-4627-402d-b66b-152f7374996a None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:45:25.993609 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:45:26.015779 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 20/38] 10.222.0.20 () {60 vars in 1131 bytes} [Tue Aug 30 15:45:25 2022] POST /identity/v3/roles => generated 225 bytes in 300 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 15:45:26.021882 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f740be-0d0b-41fd-9219-d61640cf790e None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:45:26.048181 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:45:26.053806 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c45602e-4627-402d-b66b-152f7374996a None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:45:26.098520 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5701263e-2ba6-4170-afe3-bea5f86d6e1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:45:26.099328 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5701263e-2ba6-4170-afe3-bea5f86d6e1a None admin] REQUEST_METHOD: `PUT` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:26.099616 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5701263e-2ba6-4170-afe3-bea5f86d6e1a None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:26.099892 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5701263e-2ba6-4170-afe3-bea5f86d6e1a None admin] PATH_INFO: `/v3/domains/default/users/8b9160d3cbae47a48dc9dee882da999b/roles/f265386950a641ac8413b53eda4ae34d` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:26.102667 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5701263e-2ba6-4170-afe3-bea5f86d6e1a None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=8b9160d3cbae47a48dc9dee882da999b, role_id=f265386950a641ac8413b53eda4ae34d)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:45:26.104018 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5701263e-2ba6-4170-afe3-bea5f86d6e1a None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:45:26.125683 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 19/39] 10.222.0.20 () {58 vars in 1361 bytes} [Tue Aug 30 15:45:25 2022] PUT /identity/v3/domains/default/users/8b9160d3cbae47a48dc9dee882da999b/roles/f265386950a641ac8413b53eda4ae34d => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:45:26.125683 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8f49e8-e284-4c3d-a4fe-26c98c134847 None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:26.125683 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8f49e8-e284-4c3d-a4fe-26c98c134847 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:26.126028 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8f49e8-e284-4c3d-a4fe-26c98c134847 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:26.147689 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f740be-0d0b-41fd-9219-d61640cf790e None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:45:26.154231 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-ed8f49e8-e284-4c3d-a4fe-26c98c134847 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:45:26.156087 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-ed8f49e8-e284-4c3d-a4fe-26c98c134847 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:45:26.194608 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b56931a-a678-4b47-89cb-cf592929cc79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:45:26.195494 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b56931a-a678-4b47-89cb-cf592929cc79 None admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:26.195975 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b56931a-a678-4b47-89cb-cf592929cc79 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:26.196386 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b56931a-a678-4b47-89cb-cf592929cc79 None admin] PATH_INFO: `/v3/projects` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:26.197330 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b56931a-a678-4b47-89cb-cf592929cc79 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:45:26.199159 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b56931a-a678-4b47-89cb-cf592929cc79 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:45:26.248246 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 21/40] 10.222.0.20 () {60 vars in 1141 bytes} [Tue Aug 30 15:45:26 2022] POST /identity/v3/projects => generated 301 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 15:45:26.249721 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c03f446-44b8-4695-b341-efa0c4051dc0 None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:26.250076 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c03f446-44b8-4695-b341-efa0c4051dc0 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:26.250352 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c03f446-44b8-4695-b341-efa0c4051dc0 None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:26.251835 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 22/41] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 30 15:45:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 15:45:26.257651 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585a710f-4ff1-4aff-a81a-1364c047f8e0 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:26.258301 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585a710f-4ff1-4aff-a81a-1364c047f8e0 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:26.258826 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585a710f-4ff1-4aff-a81a-1364c047f8e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:26.260245 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 20/42] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 30 15:45:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 15:45:26.298497 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-585a710f-4ff1-4aff-a81a-1364c047f8e0 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:45:26.300617 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-585a710f-4ff1-4aff-a81a-1364c047f8e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:45:26.380703 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:45:26.393401 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 23/43] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 30 15:45:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 15:45:26.531761 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8f49e8-e284-4c3d-a4fe-26c98c134847 None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:45:26.579337 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:45:26.638448 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8f49e8-e284-4c3d-a4fe-26c98c134847 None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:45:26.662216 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585a710f-4ff1-4aff-a81a-1364c047f8e0 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:45:26.774241 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585a710f-4ff1-4aff-a81a-1364c047f8e0 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:45:26.787710 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a36a32d-1ac8-4198-a9d8-29639f7571ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:45:26.788282 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a36a32d-1ac8-4198-a9d8-29639f7571ec None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:26.788562 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a36a32d-1ac8-4198-a9d8-29639f7571ec None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:26.788831 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a36a32d-1ac8-4198-a9d8-29639f7571ec None admin] PATH_INFO: `/v3/domains/default` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:26.789954 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a36a32d-1ac8-4198-a9d8-29639f7571ec None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:45:26.791417 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a36a32d-1ac8-4198-a9d8-29639f7571ec None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:45:26.793164 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 21/44] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 30 15:45:26 2022] GET /identity/v3/domains/default => generated 199 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:45:26.799253 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a36a32d-1ac8-4198-a9d8-29639f7571ec None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:45:26.811086 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646d8d6b-084c-4ef5-9503-f8e88bb0d590 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:45:26.812098 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646d8d6b-084c-4ef5-9503-f8e88bb0d590 None admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:26.812383 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646d8d6b-084c-4ef5-9503-f8e88bb0d590 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:26.812954 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646d8d6b-084c-4ef5-9503-f8e88bb0d590 None admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:26.813698 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646d8d6b-084c-4ef5-9503-f8e88bb0d590 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:45:26.815501 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646d8d6b-084c-4ef5-9503-f8e88bb0d590 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:45:26.835164 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 24/45] 10.222.0.20 () {60 vars in 1131 bytes} [Tue Aug 30 15:45:26 2022] POST /identity/v3/roles => generated 223 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 15:45:26.912465 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a36a32d-1ac8-4198-a9d8-29639f7571ec None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:45:26.927995 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:45:26.953620 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b594a63-7a04-49f0-901d-0fd3ed36faf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:45:26.954251 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b594a63-7a04-49f0-901d-0fd3ed36faf6 None admin] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:26.954539 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b594a63-7a04-49f0-901d-0fd3ed36faf6 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:26.954799 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b594a63-7a04-49f0-901d-0fd3ed36faf6 None admin] PATH_INFO: `/v3/projects` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:26.955467 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b594a63-7a04-49f0-901d-0fd3ed36faf6 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:45:26.957034 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b594a63-7a04-49f0-901d-0fd3ed36faf6 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:45:26.990585 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 22/46] 10.222.0.20 () {60 vars in 1141 bytes} [Tue Aug 30 15:45:26 2022] POST /identity/v3/projects => generated 312 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 15:45:28.112893 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135911df-1e5d-4c7f-8e46-c6d43f458783 None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:28.113438 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135911df-1e5d-4c7f-8e46-c6d43f458783 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:28.113438 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135911df-1e5d-4c7f-8e46-c6d43f458783 None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:28.114765 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 25/47] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 30 15:45:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 15:45:28.120078 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cef88d3-4cb6-401f-a525-b22aca0b8632 None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:28.120499 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cef88d3-4cb6-401f-a525-b22aca0b8632 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:28.120632 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cef88d3-4cb6-401f-a525-b22aca0b8632 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:28.140086 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-6cef88d3-4cb6-401f-a525-b22aca0b8632 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:45:28.141385 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-6cef88d3-4cb6-401f-a525-b22aca0b8632 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:45:28.209392 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 23/48] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 30 15:45:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 15:45:28.471719 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135911df-1e5d-4c7f-8e46-c6d43f458783 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:45:28.533426 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:45:28.578725 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135911df-1e5d-4c7f-8e46-c6d43f458783 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:45:28.613120 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4673ec0c-c934-41d9-b13c-34953966d612 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:45:28.613781 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4673ec0c-c934-41d9-b13c-34953966d612 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:28.614061 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4673ec0c-c934-41d9-b13c-34953966d612 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:28.614335 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4673ec0c-c934-41d9-b13c-34953966d612 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:28.630155 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4673ec0c-c934-41d9-b13c-34953966d612 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 15:45:28.636283 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-4673ec0c-c934-41d9-b13c-34953966d612 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 15:45:28.636283 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:45:28.636283 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:45:28.636283 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:45:28.636283 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:45:28.636283 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:45:28.636283 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:45:28.636283 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:45:28.636283 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:45:28.636283 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:45:28.636283 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:45:28.636283 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:45:28.636283 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 15:45:28.636283 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 15:45:28.636283 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 15:45:28.636283 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 15:45:28.637163 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 15:45:28.637163 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 15:45:28.637163 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 15:45:28.637163 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 15:45:28.637163 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:45:28.637163 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:45:28.637163 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:45:28.637163 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:45:28.637163 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:45:28.637163 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:45:28.637163 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:45:28.637163 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Aug 30 15:45:28.637163 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:45:28.637163 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:45:28.637163 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:45:28.637163 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:45:28.638189 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:45:28.638189 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:45:28.638189 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:45:28.638189 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:45:28.638189 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 15:45:28.638189 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 15:45:28.638189 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 15:45:28.638189 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 15:45:28.638189 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 15:45:28.638189 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 15:45:28.638189 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 15:45:28.638189 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Aug 30 15:45:28.638189 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 26/49] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Aug 30 15:45:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 15:45:28.643676 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cef88d3-4cb6-401f-a525-b22aca0b8632 None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:45:28.738652 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:45:28.746065 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cef88d3-4cb6-401f-a525-b22aca0b8632 None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:45:28.779828 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c397a3-d7a5-442b-bbf6-b5ad736bb88c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:45:28.780436 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c397a3-d7a5-442b-bbf6-b5ad736bb88c None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:28.780718 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c397a3-d7a5-442b-bbf6-b5ad736bb88c None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:28.780994 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c397a3-d7a5-442b-bbf6-b5ad736bb88c None admin] PATH_INFO: `/v3/roles` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:28.781757 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c397a3-d7a5-442b-bbf6-b5ad736bb88c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81512) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:45:28.782070 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c397a3-d7a5-442b-bbf6-b5ad736bb88c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:45:28.783482 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c397a3-d7a5-442b-bbf6-b5ad736bb88c None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:45:28.791583 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 24/50] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Aug 30 15:45:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 15:45:28.797480 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4673ec0c-c934-41d9-b13c-34953966d612 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:45:28.866840 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190bc1ce-092b-4264-a358-0b62341ca523 None None] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:28.867121 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190bc1ce-092b-4264-a358-0b62341ca523 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:28.867391 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190bc1ce-092b-4264-a358-0b62341ca523 None None] PATH_INFO: `/` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:28.868755 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 25/51] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 30 15:45:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 15:45:28.874151 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb1829e-70fb-40c3-9360-4351ccd74134 None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:28.874440 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb1829e-70fb-40c3-9360-4351ccd74134 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:28.874721 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb1829e-70fb-40c3-9360-4351ccd74134 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:28.895680 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-6cb1829e-70fb-40c3-9360-4351ccd74134 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:45:28.896865 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-6cb1829e-70fb-40c3-9360-4351ccd74134 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:45:28.910313 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4673ec0c-c934-41d9-b13c-34953966d612 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:45:28.927946 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:45:28.946374 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80e7b5d-22e3-4d54-aac1-284e393a2a70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:45:28.947016 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80e7b5d-22e3-4d54-aac1-284e393a2a70 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:28.947294 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80e7b5d-22e3-4d54-aac1-284e393a2a70 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:28.947566 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80e7b5d-22e3-4d54-aac1-284e393a2a70 None admin] PATH_INFO: `/v3/users/8b9160d3cbae47a48dc9dee882da999b` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:28.949639 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80e7b5d-22e3-4d54-aac1-284e393a2a70 None admin] RBAC: Authorizing `identity:get_user(user_id=8b9160d3cbae47a48dc9dee882da999b)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:45:28.951430 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80e7b5d-22e3-4d54-aac1-284e393a2a70 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:45:28.953928 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 27/52] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Aug 30 15:45:28 2022] GET /identity/v3/users/8b9160d3cbae47a48dc9dee882da999b => generated 247 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:45:28.959467 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80e7b5d-22e3-4d54-aac1-284e393a2a70 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:45:28.965466 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 26/53] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 30 15:45:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 15:45:29.056354 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80e7b5d-22e3-4d54-aac1-284e393a2a70 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:45:29.089075 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9500a53c-f1da-4c9f-a00c-8f167ad68989 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:45:29.089737 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9500a53c-f1da-4c9f-a00c-8f167ad68989 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:29.090022 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9500a53c-f1da-4c9f-a00c-8f167ad68989 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:29.090294 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9500a53c-f1da-4c9f-a00c-8f167ad68989 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:29.091462 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9500a53c-f1da-4c9f-a00c-8f167ad68989 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:45:29.092930 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9500a53c-f1da-4c9f-a00c-8f167ad68989 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:45:29.094734 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 28/54] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 30 15:45:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 15:45:29.100601 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb1829e-70fb-40c3-9360-4351ccd74134 None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:45:29.183260 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:45:29.199380 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb1829e-70fb-40c3-9360-4351ccd74134 None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:45:29.225299 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9500a53c-f1da-4c9f-a00c-8f167ad68989 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:45:29.233917 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8e6c3e-6a6f-446c-963d-00c2d3b4dd3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:45:29.234575 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8e6c3e-6a6f-446c-963d-00c2d3b4dd3e None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:29.234910 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8e6c3e-6a6f-446c-963d-00c2d3b4dd3e None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:29.235182 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8e6c3e-6a6f-446c-963d-00c2d3b4dd3e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:29.235918 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8e6c3e-6a6f-446c-963d-00c2d3b4dd3e None admin] RBAC: Adding query filter params (role.id=f265386950a641ac8413b53eda4ae34d, scope.domain.id=default, user.id=8b9160d3cbae47a48dc9dee882da999b) {{(pid=81512) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:45:29.236220 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8e6c3e-6a6f-446c-963d-00c2d3b4dd3e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:45:29.237728 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8e6c3e-6a6f-446c-963d-00c2d3b4dd3e None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:45:29.249266 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 27/55] 10.222.0.20 () {56 vars in 1322 bytes} [Tue Aug 30 15:45:29 2022] GET /identity/v3/role_assignments?user.id=8b9160d3cbae47a48dc9dee882da999b&scope.domain.id=default&role.id=f265386950a641ac8413b53eda4ae34d => generated 530 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:45:29.285376 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:45:29.446762 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9500a53c-f1da-4c9f-a00c-8f167ad68989 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:45:29.483097 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601f75f9-2054-4245-ac12-a433c098c47d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:45:29.483742 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601f75f9-2054-4245-ac12-a433c098c47d None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:29.484023 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601f75f9-2054-4245-ac12-a433c098c47d None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:29.484296 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601f75f9-2054-4245-ac12-a433c098c47d None admin] PATH_INFO: `/v3/domains/default` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:29.485617 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601f75f9-2054-4245-ac12-a433c098c47d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:45:29.487095 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601f75f9-2054-4245-ac12-a433c098c47d None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:45:29.488732 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 29/56] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 30 15:45:29 2022] GET /identity/v3/domains/default => generated 199 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:45:29.494549 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8e6c3e-6a6f-446c-963d-00c2d3b4dd3e None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:45:29.597504 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8e6c3e-6a6f-446c-963d-00c2d3b4dd3e None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:45:29.634456 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9dd6a1-fc90-4734-8c44-fc078473c437 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:45:29.635067 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9dd6a1-fc90-4734-8c44-fc078473c437 None admin] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:29.635350 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9dd6a1-fc90-4734-8c44-fc078473c437 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:29.635622 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9dd6a1-fc90-4734-8c44-fc078473c437 None admin] PATH_INFO: `/v3/projects` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:29.636285 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9dd6a1-fc90-4734-8c44-fc078473c437 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:45:29.637776 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9dd6a1-fc90-4734-8c44-fc078473c437 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:45:29.671293 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 28/57] 10.222.0.20 () {60 vars in 1140 bytes} [Tue Aug 30 15:45:29 2022] POST /identity/v3/projects => generated 298 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 15:45:31.569497 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63acb224-0cc3-4bf7-99e4-d79a575b8edf None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:31.570338 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63acb224-0cc3-4bf7-99e4-d79a575b8edf None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:31.570338 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63acb224-0cc3-4bf7-99e4-d79a575b8edf None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:31.572533 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 30/58] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 30 15:45:31 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 15:45:31.578754 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7009e3de-2af2-445c-88cf-4a701fea1eb8 None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:31.579183 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7009e3de-2af2-445c-88cf-4a701fea1eb8 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:31.579325 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7009e3de-2af2-445c-88cf-4a701fea1eb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:31.589979 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:45:31.605660 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-7009e3de-2af2-445c-88cf-4a701fea1eb8 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:45:31.606882 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-7009e3de-2af2-445c-88cf-4a701fea1eb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:45:31.677430 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 29/59] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 30 15:45:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 15:45:31.939606 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63acb224-0cc3-4bf7-99e4-d79a575b8edf None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:45:32.047793 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63acb224-0cc3-4bf7-99e4-d79a575b8edf None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:45:32.071723 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:45:32.084854 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce8cc94-77b5-4f34-a654-e08d0c17af27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:45:32.085522 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce8cc94-77b5-4f34-a654-e08d0c17af27 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:32.085805 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce8cc94-77b5-4f34-a654-e08d0c17af27 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:32.086080 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce8cc94-77b5-4f34-a654-e08d0c17af27 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:32.087221 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce8cc94-77b5-4f34-a654-e08d0c17af27 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:45:32.088677 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce8cc94-77b5-4f34-a654-e08d0c17af27 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:45:32.090307 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 31/60] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 30 15:45:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:45:32.096174 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7009e3de-2af2-445c-88cf-4a701fea1eb8 None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:45:32.210702 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7009e3de-2af2-445c-88cf-4a701fea1eb8 None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:45:32.240494 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:45:32.246308 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b2a92b-fde2-419e-92c9-3b963002dbec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:45:32.247271 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b2a92b-fde2-419e-92c9-3b963002dbec None admin] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:32.247553 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b2a92b-fde2-419e-92c9-3b963002dbec None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:32.247823 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b2a92b-fde2-419e-92c9-3b963002dbec None admin] PATH_INFO: `/v3/users` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:32.248473 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b2a92b-fde2-419e-92c9-3b963002dbec None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:45:32.250037 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b2a92b-fde2-419e-92c9-3b963002dbec None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:45:32.279160 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 30/61] 10.222.0.20 () {60 vars in 1132 bytes} [Tue Aug 30 15:45:32 2022] POST /identity/v3/users => generated 275 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 15:45:35.176332 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf962f8-5aa3-4a00-8b7a-56fc39a84cd2 None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:35.176854 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf962f8-5aa3-4a00-8b7a-56fc39a84cd2 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:35.176929 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf962f8-5aa3-4a00-8b7a-56fc39a84cd2 None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:35.178407 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 32/62] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 30 15:45:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 15:45:35.183373 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4483bdef-4d96-43b3-913e-2defef5c5b2e None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:35.183753 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4483bdef-4d96-43b3-913e-2defef5c5b2e None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:35.183942 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4483bdef-4d96-43b3-913e-2defef5c5b2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:35.210212 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-4483bdef-4d96-43b3-913e-2defef5c5b2e None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:45:35.211472 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-4483bdef-4d96-43b3-913e-2defef5c5b2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:45:35.294946 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f7f4fa-9e4c-4c22-9b7c-6a0749e6d32e None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:35.295269 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f7f4fa-9e4c-4c22-9b7c-6a0749e6d32e None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:35.295495 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f7f4fa-9e4c-4c22-9b7c-6a0749e6d32e None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:35.297187 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 33/63] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 30 15:45:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 15:45:35.302119 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685022f8-a4f4-4d35-bad4-efd51ef6b456 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:35.302653 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 31/64] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 30 15:45:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 15:45:35.302902 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685022f8-a4f4-4d35-bad4-efd51ef6b456 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:35.307346 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685022f8-a4f4-4d35-bad4-efd51ef6b456 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:35.346299 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-685022f8-a4f4-4d35-bad4-efd51ef6b456 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:45:35.347547 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-685022f8-a4f4-4d35-bad4-efd51ef6b456 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:45:35.439643 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 34/65] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 30 15:45:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 15:45:35.517820 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a123db6f-1c4b-4237-98ac-04c03696aa3b None None] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:35.518461 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a123db6f-1c4b-4237-98ac-04c03696aa3b None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:35.518930 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a123db6f-1c4b-4237-98ac-04c03696aa3b None None] PATH_INFO: `/` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:35.520528 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 32/66] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 30 15:45:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 15:45:35.526092 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196eeede-11cb-4183-8c77-b1038ac2dbd2 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:35.526615 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196eeede-11cb-4183-8c77-b1038ac2dbd2 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:35.527090 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196eeede-11cb-4183-8c77-b1038ac2dbd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:35.545974 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-196eeede-11cb-4183-8c77-b1038ac2dbd2 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:45:35.547531 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-196eeede-11cb-4183-8c77-b1038ac2dbd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:45:35.616143 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 35/67] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 30 15:45:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 15:45:35.679509 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a123db6f-1c4b-4237-98ac-04c03696aa3b None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:45:35.807418 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a123db6f-1c4b-4237-98ac-04c03696aa3b None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:45:35.819369 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196eeede-11cb-4183-8c77-b1038ac2dbd2 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:45:35.843063 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:45:35.871217 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861ac843-7374-4427-92f8-d90f376a27d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:45:35.872209 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861ac843-7374-4427-92f8-d90f376a27d2 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:35.872604 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861ac843-7374-4427-92f8-d90f376a27d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:35.872880 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861ac843-7374-4427-92f8-d90f376a27d2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:35.884681 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861ac843-7374-4427-92f8-d90f376a27d2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 15:45:35.889644 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application [None req-861ac843-7374-4427-92f8-d90f376a27d2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 30 15:45:35.889644 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:45:35.889644 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:45:35.889644 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:45:35.889644 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:45:35.889644 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:45:35.889644 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:45:35.889644 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:45:35.889644 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:45:35.889644 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:45:35.889644 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:45:35.889644 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:45:35.889644 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 15:45:35.889644 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 15:45:35.889644 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 15:45:35.889644 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 15:45:35.890589 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 15:45:35.890589 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 15:45:35.890589 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 15:45:35.890589 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 15:45:35.890589 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:45:35.890589 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:45:35.890589 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:45:35.890589 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:45:35.890589 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:45:35.890589 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:45:35.890589 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:45:35.890589 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application with Lock( Aug 30 15:45:35.890589 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:45:35.890589 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:45:35.890589 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:45:35.890589 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:45:35.891360 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:45:35.891360 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:45:35.891360 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:45:35.891360 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:45:35.891360 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 15:45:35.891360 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 15:45:35.891360 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 15:45:35.891360 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 15:45:35.891360 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 15:45:35.891360 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 15:45:35.891360 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 30 15:45:35.891360 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Aug 30 15:45:35.891360 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 33/68] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Aug 30 15:45:35 2022] GET /identity/v3/roles/member => generated 84 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 15:45:35.901568 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861ac843-7374-4427-92f8-d90f376a27d2 None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:45:35.920977 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:45:35.989911 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196eeede-11cb-4183-8c77-b1038ac2dbd2 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:45:36.025961 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd4803a-d441-47a5-9ef0-8779fd849535 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:45:36.026602 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd4803a-d441-47a5-9ef0-8779fd849535 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:36.026883 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd4803a-d441-47a5-9ef0-8779fd849535 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:36.027155 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd4803a-d441-47a5-9ef0-8779fd849535 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:36.034036 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861ac843-7374-4427-92f8-d90f376a27d2 None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:45:36.038541 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd4803a-d441-47a5-9ef0-8779fd849535 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 15:45:36.043620 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-2bd4803a-d441-47a5-9ef0-8779fd849535 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 15:45:36.043620 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:45:36.043620 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:45:36.043620 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:45:36.043620 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:45:36.043620 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:45:36.043620 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:45:36.043620 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:45:36.043620 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:45:36.043620 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:45:36.043620 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:45:36.043620 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:45:36.043620 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 15:45:36.043620 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 15:45:36.043620 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 15:45:36.043620 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 15:45:36.044506 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 15:45:36.044506 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 15:45:36.044506 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 15:45:36.044506 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 15:45:36.044506 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:45:36.044506 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:45:36.044506 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:45:36.044506 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:45:36.044506 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:45:36.044506 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:45:36.044506 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:45:36.044506 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Aug 30 15:45:36.044506 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:45:36.044506 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:45:36.044506 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:45:36.044506 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:45:36.045310 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:45:36.045310 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:45:36.045310 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:45:36.045310 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:45:36.045310 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 15:45:36.045310 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 15:45:36.045310 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 15:45:36.045310 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 15:45:36.045310 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 15:45:36.045310 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 15:45:36.045310 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 15:45:36.045310 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Aug 30 15:45:36.045310 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 36/69] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Aug 30 15:45:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 15:45:36.046423 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd4803a-d441-47a5-9ef0-8779fd849535 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:45:36.074827 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ce8175-08ef-42d9-bee4-3fdef1bbcb8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:45:36.075367 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ce8175-08ef-42d9-bee4-3fdef1bbcb8f None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:36.075653 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ce8175-08ef-42d9-bee4-3fdef1bbcb8f None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:36.075930 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ce8175-08ef-42d9-bee4-3fdef1bbcb8f None admin] PATH_INFO: `/v3/roles` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:36.076611 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ce8175-08ef-42d9-bee4-3fdef1bbcb8f None admin] RBAC: Adding query filter params (name=member) {{(pid=81512) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:45:36.076922 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ce8175-08ef-42d9-bee4-3fdef1bbcb8f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:45:36.078418 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ce8175-08ef-42d9-bee4-3fdef1bbcb8f None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:45:36.087202 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 34/70] 10.222.0.20 () {56 vars in 1101 bytes} [Tue Aug 30 15:45:35 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:45:36.088396 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cba57a1-d73e-4d0f-9cf4-66af594405ba None None] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:36.088687 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cba57a1-d73e-4d0f-9cf4-66af594405ba None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:36.088963 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cba57a1-d73e-4d0f-9cf4-66af594405ba None None] PATH_INFO: `/` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:36.090223 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 35/71] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 30 15:45:36 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 15:45:36.090899 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cba57a1-d73e-4d0f-9cf4-66af594405ba None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:45:36.132956 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:45:36.149675 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd4803a-d441-47a5-9ef0-8779fd849535 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:45:36.160654 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:45:36.184420 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa6d42b-50f8-4a09-8f0f-3af82bd1c62e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:45:36.185060 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa6d42b-50f8-4a09-8f0f-3af82bd1c62e None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:36.185365 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa6d42b-50f8-4a09-8f0f-3af82bd1c62e None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:36.185636 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa6d42b-50f8-4a09-8f0f-3af82bd1c62e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:36.191325 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cba57a1-d73e-4d0f-9cf4-66af594405ba None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:45:36.195381 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa6d42b-50f8-4a09-8f0f-3af82bd1c62e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 15:45:36.200205 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-daa6d42b-50f8-4a09-8f0f-3af82bd1c62e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 15:45:36.200205 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:45:36.200205 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:45:36.200205 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:45:36.200205 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:45:36.200205 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:45:36.200205 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:45:36.200205 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:45:36.200205 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:45:36.200205 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:45:36.200205 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:45:36.200205 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:45:36.200205 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 15:45:36.200205 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 15:45:36.200205 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 15:45:36.200205 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 15:45:36.201062 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 15:45:36.201062 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 15:45:36.201062 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 15:45:36.201062 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 15:45:36.201062 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:45:36.201062 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:45:36.201062 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:45:36.201062 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:45:36.201062 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:45:36.201062 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:45:36.201062 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:45:36.201062 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Aug 30 15:45:36.201062 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:45:36.201062 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:45:36.201062 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:45:36.201062 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:45:36.202004 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:45:36.202004 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:45:36.202004 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:45:36.202004 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:45:36.202004 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 15:45:36.202004 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 15:45:36.202004 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 15:45:36.202004 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 15:45:36.202004 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 15:45:36.202004 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 15:45:36.202004 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 15:45:36.202004 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Aug 30 15:45:36.202004 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 37/72] 10.222.0.20 () {56 vars in 1114 bytes} [Tue Aug 30 15:45:36 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 15:45:36.202917 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd66d744-9716-48a6-9c16-77ef10ee4776 None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:36.203213 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd66d744-9716-48a6-9c16-77ef10ee4776 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:36.203506 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd66d744-9716-48a6-9c16-77ef10ee4776 None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:36.204751 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 38/73] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 30 15:45:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 15:45:36.207172 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd66d744-9716-48a6-9c16-77ef10ee4776 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:45:36.227029 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a4d513-e48e-4c6f-892f-e15b97aefa2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:45:36.227635 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a4d513-e48e-4c6f-892f-e15b97aefa2e None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:36.227918 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a4d513-e48e-4c6f-892f-e15b97aefa2e None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:36.228187 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a4d513-e48e-4c6f-892f-e15b97aefa2e None admin] PATH_INFO: `/v3/roles` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:36.228842 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a4d513-e48e-4c6f-892f-e15b97aefa2e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81512) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:45:36.229158 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a4d513-e48e-4c6f-892f-e15b97aefa2e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:45:36.230611 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a4d513-e48e-4c6f-892f-e15b97aefa2e None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:45:36.238917 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 36/74] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Aug 30 15:45:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:45:36.240056 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9855c2e5-7a79-4818-8b77-b899e5455dd0 None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:36.241013 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9855c2e5-7a79-4818-8b77-b899e5455dd0 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:36.241013 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9855c2e5-7a79-4818-8b77-b899e5455dd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:36.258515 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-9855c2e5-7a79-4818-8b77-b899e5455dd0 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:45:36.259419 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-9855c2e5-7a79-4818-8b77-b899e5455dd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:45:36.306079 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd66d744-9716-48a6-9c16-77ef10ee4776 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:45:36.327296 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 37/75] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 30 15:45:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 15:45:36.328149 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9855c2e5-7a79-4818-8b77-b899e5455dd0 None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:45:36.340299 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5da96b-20ea-40c4-9a57-985bcac675b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:45:36.340933 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5da96b-20ea-40c4-9a57-985bcac675b0 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:36.341242 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5da96b-20ea-40c4-9a57-985bcac675b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:36.341529 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5da96b-20ea-40c4-9a57-985bcac675b0 None admin] PATH_INFO: `/v3/users/8249204af55844a180f2d1d09c8b1f59` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:36.357958 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5da96b-20ea-40c4-9a57-985bcac675b0 None admin] RBAC: Authorizing `identity:get_user(user_id=8249204af55844a180f2d1d09c8b1f59)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:45:36.359473 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5da96b-20ea-40c4-9a57-985bcac675b0 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:45:36.361341 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 39/76] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Aug 30 15:45:36 2022] GET /identity/v3/users/8249204af55844a180f2d1d09c8b1f59 => generated 275 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:45:36.367199 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2add59-9aa8-45a2-af7f-4be9a2e82ffa None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:36.367580 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2add59-9aa8-45a2-af7f-4be9a2e82ffa None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:36.368097 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2add59-9aa8-45a2-af7f-4be9a2e82ffa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:36.373404 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:45:36.389477 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-ca2add59-9aa8-45a2-af7f-4be9a2e82ffa None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:45:36.390859 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-ca2add59-9aa8-45a2-af7f-4be9a2e82ffa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:45:36.418403 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:45:36.446292 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9855c2e5-7a79-4818-8b77-b899e5455dd0 None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:45:36.466359 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 40/77] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 30 15:45:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 15:45:36.467099 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2add59-9aa8-45a2-af7f-4be9a2e82ffa None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:45:36.490652 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea0b9dd-62f0-4ba9-a983-b6a68c465e27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:45:36.491502 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea0b9dd-62f0-4ba9-a983-b6a68c465e27 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:36.491948 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea0b9dd-62f0-4ba9-a983-b6a68c465e27 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:36.492884 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea0b9dd-62f0-4ba9-a983-b6a68c465e27 None admin] PATH_INFO: `/v3/roles` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:36.493736 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea0b9dd-62f0-4ba9-a983-b6a68c465e27 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81512) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:45:36.494427 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea0b9dd-62f0-4ba9-a983-b6a68c465e27 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:45:36.496634 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea0b9dd-62f0-4ba9-a983-b6a68c465e27 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:45:36.509058 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 38/78] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Aug 30 15:45:36 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:45:36.514878 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea0b9dd-62f0-4ba9-a983-b6a68c465e27 None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:45:36.586050 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2add59-9aa8-45a2-af7f-4be9a2e82ffa None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:45:36.622558 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68989450-738a-49c8-aa32-7dcbb19b8c94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:45:36.623199 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68989450-738a-49c8-aa32-7dcbb19b8c94 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:36.623480 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68989450-738a-49c8-aa32-7dcbb19b8c94 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:36.623752 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68989450-738a-49c8-aa32-7dcbb19b8c94 None admin] PATH_INFO: `/v3/users/8b9160d3cbae47a48dc9dee882da999b` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:36.624918 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68989450-738a-49c8-aa32-7dcbb19b8c94 None admin] RBAC: Authorizing `identity:get_user(user_id=8b9160d3cbae47a48dc9dee882da999b)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:45:36.626878 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68989450-738a-49c8-aa32-7dcbb19b8c94 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:45:36.628660 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 41/79] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Aug 30 15:45:36 2022] GET /identity/v3/users/8b9160d3cbae47a48dc9dee882da999b => generated 247 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:45:36.629811 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68989450-738a-49c8-aa32-7dcbb19b8c94 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:45:36.632401 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:45:36.641831 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea0b9dd-62f0-4ba9-a983-b6a68c465e27 None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:45:36.666376 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:45:36.676886 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ad1ad6-65e3-4c23-a665-7e325741dded None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:45:36.677542 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ad1ad6-65e3-4c23-a665-7e325741dded None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:36.677828 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ad1ad6-65e3-4c23-a665-7e325741dded None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:36.678104 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ad1ad6-65e3-4c23-a665-7e325741dded None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:36.740619 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68989450-738a-49c8-aa32-7dcbb19b8c94 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:45:36.746550 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ad1ad6-65e3-4c23-a665-7e325741dded None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:45:36.748405 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ad1ad6-65e3-4c23-a665-7e325741dded None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:45:36.775166 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758afd23-8037-492b-8bee-376243789746 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:45:36.775801 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758afd23-8037-492b-8bee-376243789746 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:36.776083 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758afd23-8037-492b-8bee-376243789746 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:36.776357 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758afd23-8037-492b-8bee-376243789746 None admin] PATH_INFO: `/v3/users/8249204af55844a180f2d1d09c8b1f59` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:36.777532 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758afd23-8037-492b-8bee-376243789746 None admin] RBAC: Authorizing `identity:get_user(user_id=8249204af55844a180f2d1d09c8b1f59)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:45:36.778981 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758afd23-8037-492b-8bee-376243789746 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:45:36.780623 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 42/80] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Aug 30 15:45:36 2022] GET /identity/v3/users/8249204af55844a180f2d1d09c8b1f59 => generated 275 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:45:36.781464 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758afd23-8037-492b-8bee-376243789746 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:45:36.840814 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:45:36.848250 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 39/81] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Aug 30 15:45:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 15:45:36.849142 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ad1ad6-65e3-4c23-a665-7e325741dded None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:45:36.882022 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:45:36.890225 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758afd23-8037-492b-8bee-376243789746 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:45:36.925383 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15762c9-6a10-40ee-8799-9e467718645f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:45:36.926024 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15762c9-6a10-40ee-8799-9e467718645f None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:36.926657 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15762c9-6a10-40ee-8799-9e467718645f None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:36.926937 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15762c9-6a10-40ee-8799-9e467718645f None admin] PATH_INFO: `/v3/roles/member` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:36.937529 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15762c9-6a10-40ee-8799-9e467718645f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 15:45:36.944574 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-b15762c9-6a10-40ee-8799-9e467718645f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 30 15:45:36.944574 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:45:36.944574 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:45:36.944574 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:45:36.944574 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:45:36.944574 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:45:36.944574 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:45:36.944574 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:45:36.944574 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:45:36.944574 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:45:36.944574 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:45:36.944574 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:45:36.944574 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 15:45:36.944574 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 15:45:36.944574 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 15:45:36.944574 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 15:45:36.945640 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 15:45:36.945640 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 15:45:36.945640 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 15:45:36.945640 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 15:45:36.945640 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:45:36.945640 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:45:36.945640 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:45:36.945640 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:45:36.945640 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:45:36.945640 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:45:36.945640 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:45:36.945640 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Aug 30 15:45:36.945640 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:45:36.945640 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:45:36.945640 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:45:36.945640 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:45:36.946686 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:45:36.946686 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:45:36.946686 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:45:36.946686 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:45:36.946686 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 15:45:36.946686 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 15:45:36.946686 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 15:45:36.946686 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 15:45:36.946686 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 15:45:36.946686 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 15:45:36.946686 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 30 15:45:36.946686 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Aug 30 15:45:36.946686 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 43/82] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Aug 30 15:45:36 2022] GET /identity/v3/roles/member => generated 84 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 15:45:36.947429 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15762c9-6a10-40ee-8799-9e467718645f None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:45:36.954428 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ad1ad6-65e3-4c23-a665-7e325741dded None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:45:36.990876 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01905289-beaf-4f91-9a9f-66b88b4e9d7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:45:36.991490 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01905289-beaf-4f91-9a9f-66b88b4e9d7e None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:36.991769 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01905289-beaf-4f91-9a9f-66b88b4e9d7e None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:36.992041 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01905289-beaf-4f91-9a9f-66b88b4e9d7e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:37.050682 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:45:37.051232 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15762c9-6a10-40ee-8799-9e467718645f None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:45:37.058071 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01905289-beaf-4f91-9a9f-66b88b4e9d7e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:45:37.059548 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01905289-beaf-4f91-9a9f-66b88b4e9d7e None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:45:37.084198 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670c15b2-3a45-4315-9020-62e8ad59a8cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:45:37.084830 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670c15b2-3a45-4315-9020-62e8ad59a8cc None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:37.085115 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670c15b2-3a45-4315-9020-62e8ad59a8cc None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:37.085430 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670c15b2-3a45-4315-9020-62e8ad59a8cc None admin] PATH_INFO: `/v3/domains/default` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:37.086557 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670c15b2-3a45-4315-9020-62e8ad59a8cc None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:45:37.088004 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670c15b2-3a45-4315-9020-62e8ad59a8cc None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:45:37.089636 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 44/83] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 30 15:45:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:45:37.090391 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670c15b2-3a45-4315-9020-62e8ad59a8cc None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:45:37.114659 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:45:37.162076 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 40/84] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Aug 30 15:45:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 15:45:37.163292 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01905289-beaf-4f91-9a9f-66b88b4e9d7e None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:45:37.190089 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670c15b2-3a45-4315-9020-62e8ad59a8cc None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:45:37.223876 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c316b340-8ca1-47b6-8c4a-9901b9d53573 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:45:37.224622 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c316b340-8ca1-47b6-8c4a-9901b9d53573 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:37.224967 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c316b340-8ca1-47b6-8c4a-9901b9d53573 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:37.225341 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c316b340-8ca1-47b6-8c4a-9901b9d53573 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:37.252913 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:45:37.259710 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01905289-beaf-4f91-9a9f-66b88b4e9d7e None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:45:37.291983 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c316b340-8ca1-47b6-8c4a-9901b9d53573 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:45:37.293079 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62dbc20a-2dfc-4ff6-87b6-9c9b77109a22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:45:37.293506 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c316b340-8ca1-47b6-8c4a-9901b9d53573 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:45:37.293743 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dbc20a-2dfc-4ff6-87b6-9c9b77109a22 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:37.294028 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dbc20a-2dfc-4ff6-87b6-9c9b77109a22 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:37.294306 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dbc20a-2dfc-4ff6-87b6-9c9b77109a22 None admin] PATH_INFO: `/v3/projects/2a4e4930fe3247d3b04ad23013176d45` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:37.295453 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62dbc20a-2dfc-4ff6-87b6-9c9b77109a22 None admin] RBAC: Authorizing `identity:get_project(project_id=2a4e4930fe3247d3b04ad23013176d45)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:45:37.296984 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62dbc20a-2dfc-4ff6-87b6-9c9b77109a22 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:45:37.298665 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 41/85] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Aug 30 15:45:37 2022] GET /identity/v3/projects/2a4e4930fe3247d3b04ad23013176d45 => generated 298 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:45:37.299431 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62dbc20a-2dfc-4ff6-87b6-9c9b77109a22 None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:45:37.316485 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:45:37.390044 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 45/86] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Aug 30 15:45:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 15:45:37.390920 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c316b340-8ca1-47b6-8c4a-9901b9d53573 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:45:37.399327 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62dbc20a-2dfc-4ff6-87b6-9c9b77109a22 None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:45:37.432638 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b422c4f-9f33-4357-a042-3e588943efd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:45:37.433266 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b422c4f-9f33-4357-a042-3e588943efd8 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:37.433551 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b422c4f-9f33-4357-a042-3e588943efd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:37.433821 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b422c4f-9f33-4357-a042-3e588943efd8 None admin] PATH_INFO: `/v3/roles` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:37.434484 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b422c4f-9f33-4357-a042-3e588943efd8 None admin] RBAC: Adding query filter params (name=member) {{(pid=81512) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:45:37.434787 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b422c4f-9f33-4357-a042-3e588943efd8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:45:37.436188 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b422c4f-9f33-4357-a042-3e588943efd8 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:45:37.444089 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 42/87] 10.222.0.20 () {56 vars in 1101 bytes} [Tue Aug 30 15:45:37 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:45:37.444873 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b422c4f-9f33-4357-a042-3e588943efd8 None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:45:37.469736 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:45:37.493581 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c316b340-8ca1-47b6-8c4a-9901b9d53573 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:45:37.516389 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:45:37.529158 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e0192f-e6bb-45bd-9ca1-f26bfb2b5424 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:45:37.529803 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e0192f-e6bb-45bd-9ca1-f26bfb2b5424 None admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:37.530086 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e0192f-e6bb-45bd-9ca1-f26bfb2b5424 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:37.530362 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e0192f-e6bb-45bd-9ca1-f26bfb2b5424 None admin] PATH_INFO: `/v3/users` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:37.531034 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e0192f-e6bb-45bd-9ca1-f26bfb2b5424 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:45:37.532563 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e0192f-e6bb-45bd-9ca1-f26bfb2b5424 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:45:37.542875 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b422c4f-9f33-4357-a042-3e588943efd8 None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:45:37.559922 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 46/88] 10.222.0.20 () {60 vars in 1132 bytes} [Tue Aug 30 15:45:37 2022] POST /identity/v3/users => generated 289 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 15:45:37.560775 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e0192f-e6bb-45bd-9ca1-f26bfb2b5424 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:45:37.589562 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5214229-a163-455b-9450-ca2ee483e632 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:45:37.590287 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5214229-a163-455b-9450-ca2ee483e632 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:37.590615 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5214229-a163-455b-9450-ca2ee483e632 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:37.590893 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5214229-a163-455b-9450-ca2ee483e632 None admin] PATH_INFO: `/v3/projects/2a4e4930fe3247d3b04ad23013176d45` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:37.592499 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5214229-a163-455b-9450-ca2ee483e632 None admin] RBAC: Authorizing `identity:get_project(project_id=2a4e4930fe3247d3b04ad23013176d45)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:45:37.594341 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5214229-a163-455b-9450-ca2ee483e632 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:45:37.596469 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 43/89] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Aug 30 15:45:37 2022] GET /identity/v3/projects/2a4e4930fe3247d3b04ad23013176d45 => generated 298 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:45:37.598543 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5214229-a163-455b-9450-ca2ee483e632 None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:45:37.686023 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e0192f-e6bb-45bd-9ca1-f26bfb2b5424 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:45:37.711541 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:45:37.720697 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5214229-a163-455b-9450-ca2ee483e632 None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:45:37.727564 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53248409-76b0-46c4-8bca-f3677703c9f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:45:37.728605 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53248409-76b0-46c4-8bca-f3677703c9f3 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:37.728605 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53248409-76b0-46c4-8bca-f3677703c9f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:37.728773 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53248409-76b0-46c4-8bca-f3677703c9f3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:37.729494 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53248409-76b0-46c4-8bca-f3677703c9f3 None admin] RBAC: Adding query filter params (role.id=df8f9f844b1b439181d75a07c1a93418, scope.project.id=2a4e4930fe3247d3b04ad23013176d45, user.id=8249204af55844a180f2d1d09c8b1f59) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:45:37.729802 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53248409-76b0-46c4-8bca-f3677703c9f3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:45:37.731269 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53248409-76b0-46c4-8bca-f3677703c9f3 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:45:37.745999 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 47/90] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Aug 30 15:45:37 2022] GET /identity/v3/role_assignments?user.id=8249204af55844a180f2d1d09c8b1f59&scope.project.id=2a4e4930fe3247d3b04ad23013176d45&role.id=df8f9f844b1b439181d75a07c1a93418 => generated 259 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:45:37.745999 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53248409-76b0-46c4-8bca-f3677703c9f3 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:45:37.760777 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6392475-9000-4ff5-a529-09c8ac979502 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:45:37.761477 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6392475-9000-4ff5-a529-09c8ac979502 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:37.761751 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6392475-9000-4ff5-a529-09c8ac979502 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:37.762023 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6392475-9000-4ff5-a529-09c8ac979502 None admin] PATH_INFO: `/v3/projects/2a4e4930fe3247d3b04ad23013176d45` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:37.763730 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6392475-9000-4ff5-a529-09c8ac979502 None admin] RBAC: Authorizing `identity:get_project(project_id=2a4e4930fe3247d3b04ad23013176d45)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:45:37.765259 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6392475-9000-4ff5-a529-09c8ac979502 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:45:37.767280 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 44/91] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Aug 30 15:45:37 2022] GET /identity/v3/projects/2a4e4930fe3247d3b04ad23013176d45 => generated 298 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:45:37.767880 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6392475-9000-4ff5-a529-09c8ac979502 None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:45:37.818004 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:45:37.907756 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53248409-76b0-46c4-8bca-f3677703c9f3 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:45:37.921033 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6392475-9000-4ff5-a529-09c8ac979502 None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:45:37.944680 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68702d9c-2fb6-496b-b142-a737bc6b31b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:45:37.945380 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68702d9c-2fb6-496b-b142-a737bc6b31b0 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:37.945655 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68702d9c-2fb6-496b-b142-a737bc6b31b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:37.945928 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68702d9c-2fb6-496b-b142-a737bc6b31b0 None admin] PATH_INFO: `/v3/users/8249204af55844a180f2d1d09c8b1f59` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:37.947153 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68702d9c-2fb6-496b-b142-a737bc6b31b0 None admin] RBAC: Authorizing `identity:get_user(user_id=8249204af55844a180f2d1d09c8b1f59)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:45:37.948639 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68702d9c-2fb6-496b-b142-a737bc6b31b0 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:45:37.950437 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 48/92] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Aug 30 15:45:37 2022] GET /identity/v3/users/8249204af55844a180f2d1d09c8b1f59 => generated 275 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:45:37.954444 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68702d9c-2fb6-496b-b142-a737bc6b31b0 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:45:37.967985 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d28163a-6821-4021-9740-6c7e83bc942a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:45:37.969334 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d28163a-6821-4021-9740-6c7e83bc942a None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:37.969964 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d28163a-6821-4021-9740-6c7e83bc942a None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:37.970481 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d28163a-6821-4021-9740-6c7e83bc942a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:37.971359 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d28163a-6821-4021-9740-6c7e83bc942a None admin] RBAC: Adding query filter params (role.id=f265386950a641ac8413b53eda4ae34d, scope.project.id=2a4e4930fe3247d3b04ad23013176d45, user.id=8b9160d3cbae47a48dc9dee882da999b) {{(pid=81512) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:45:37.972051 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d28163a-6821-4021-9740-6c7e83bc942a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:45:37.974580 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d28163a-6821-4021-9740-6c7e83bc942a None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:45:37.983877 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 45/93] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Aug 30 15:45:37 2022] GET /identity/v3/role_assignments?user.id=8b9160d3cbae47a48dc9dee882da999b&scope.project.id=2a4e4930fe3247d3b04ad23013176d45&role.id=f265386950a641ac8413b53eda4ae34d => generated 259 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:45:37.985043 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d28163a-6821-4021-9740-6c7e83bc942a None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:45:38.036225 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:45:38.152727 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d28163a-6821-4021-9740-6c7e83bc942a None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:45:38.231566 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bda65d-34a8-48d4-b201-e4587af9a6b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:45:38.232118 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bda65d-34a8-48d4-b201-e4587af9a6b5 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:38.232399 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bda65d-34a8-48d4-b201-e4587af9a6b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:38.232711 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bda65d-34a8-48d4-b201-e4587af9a6b5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:38.235529 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68702d9c-2fb6-496b-b142-a737bc6b31b0 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:45:38.259081 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:45:38.330052 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3bda65d-34a8-48d4-b201-e4587af9a6b5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:45:38.332404 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3bda65d-34a8-48d4-b201-e4587af9a6b5 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:45:38.350046 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685a751e-e36f-494d-89ad-e400acce59f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:45:38.350696 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685a751e-e36f-494d-89ad-e400acce59f3 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:38.350979 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685a751e-e36f-494d-89ad-e400acce59f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:38.351249 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685a751e-e36f-494d-89ad-e400acce59f3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:38.352008 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685a751e-e36f-494d-89ad-e400acce59f3 None admin] RBAC: Adding query filter params (role.id=6b4385c3b449456d848c9abb16766d25, scope.project.id=2a4e4930fe3247d3b04ad23013176d45, user.id=8249204af55844a180f2d1d09c8b1f59) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:45:38.352317 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685a751e-e36f-494d-89ad-e400acce59f3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:45:38.353838 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685a751e-e36f-494d-89ad-e400acce59f3 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:45:38.373589 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:45:38.375010 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 49/94] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Aug 30 15:45:37 2022] GET /identity/v3/role_assignments?user.id=8249204af55844a180f2d1d09c8b1f59&scope.project.id=2a4e4930fe3247d3b04ad23013176d45&role.id=6b4385c3b449456d848c9abb16766d25 => generated 259 bytes in 424 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:45:38.499225 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 46/95] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Aug 30 15:45:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 515 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 15:45:38.505422 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685a751e-e36f-494d-89ad-e400acce59f3 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:45:38.659031 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685a751e-e36f-494d-89ad-e400acce59f3 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:45:38.704790 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28a807c-be35-49b3-823e-4c4505fd6420 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:45:38.705525 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28a807c-be35-49b3-823e-4c4505fd6420 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:38.705811 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28a807c-be35-49b3-823e-4c4505fd6420 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:38.706092 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28a807c-be35-49b3-823e-4c4505fd6420 None admin] PATH_INFO: `/v3/projects/8ed1a9892a9a428a92a900c9ce92f2b0` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:38.707243 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28a807c-be35-49b3-823e-4c4505fd6420 None admin] RBAC: Authorizing `identity:get_project(project_id=8ed1a9892a9a428a92a900c9ce92f2b0)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:45:38.708705 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28a807c-be35-49b3-823e-4c4505fd6420 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:45:38.710404 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 50/96] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Aug 30 15:45:38 2022] GET /identity/v3/projects/8ed1a9892a9a428a92a900c9ce92f2b0 => generated 312 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:45:38.716503 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bda65d-34a8-48d4-b201-e4587af9a6b5 None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:45:38.840132 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bda65d-34a8-48d4-b201-e4587af9a6b5 None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:45:38.869108 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:45:38.885885 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a12f967-fab4-4829-979c-d84ebed9b9f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:45:38.886512 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a12f967-fab4-4829-979c-d84ebed9b9f7 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:38.886794 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a12f967-fab4-4829-979c-d84ebed9b9f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:38.887128 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a12f967-fab4-4829-979c-d84ebed9b9f7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:38.887869 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a12f967-fab4-4829-979c-d84ebed9b9f7 None admin] RBAC: Adding query filter params (role.id=df8f9f844b1b439181d75a07c1a93418, scope.project.id=8ed1a9892a9a428a92a900c9ce92f2b0, user.id=8249204af55844a180f2d1d09c8b1f59) {{(pid=81512) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:45:38.888179 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a12f967-fab4-4829-979c-d84ebed9b9f7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:45:38.889672 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a12f967-fab4-4829-979c-d84ebed9b9f7 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:45:38.895426 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 47/97] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Aug 30 15:45:38 2022] GET /identity/v3/role_assignments?user.id=8249204af55844a180f2d1d09c8b1f59&scope.project.id=8ed1a9892a9a428a92a900c9ce92f2b0&role.id=df8f9f844b1b439181d75a07c1a93418 => generated 259 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:45:40.401631 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c736a3bc-19a6-404f-abd1-03b93b64c743 None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:40.402173 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c736a3bc-19a6-404f-abd1-03b93b64c743 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:40.402173 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c736a3bc-19a6-404f-abd1-03b93b64c743 None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:40.403685 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 51/98] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 30 15:45:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 15:45:40.408817 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9b901c-c009-40d8-a740-825899678064 None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:40.409211 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9b901c-c009-40d8-a740-825899678064 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:40.409399 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9b901c-c009-40d8-a740-825899678064 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:40.433695 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493c7e97-13b8-4192-8e11-805e2ae2e334 None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:40.434016 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493c7e97-13b8-4192-8e11-805e2ae2e334 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:40.434241 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493c7e97-13b8-4192-8e11-805e2ae2e334 None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:40.435594 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 52/99] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 30 15:45:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 15:45:40.436384 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-2c9b901c-c009-40d8-a740-825899678064 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:45:40.437791 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-2c9b901c-c009-40d8-a740-825899678064 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:45:40.440670 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84e8759-1c99-44e9-bfae-8fef0e4c54d0 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:40.440950 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84e8759-1c99-44e9-bfae-8fef0e4c54d0 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:40.441238 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84e8759-1c99-44e9-bfae-8fef0e4c54d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:40.467243 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-d84e8759-1c99-44e9-bfae-8fef0e4c54d0 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:45:40.468450 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-d84e8759-1c99-44e9-bfae-8fef0e4c54d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:45:40.537530 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:45:40.579836 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 53/100] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 30 15:45:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Aug 30 15:45:40.581975 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348c2f85-b4b9-4a5c-9545-881461cbddf3 None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:40.582269 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348c2f85-b4b9-4a5c-9545-881461cbddf3 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:40.582538 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348c2f85-b4b9-4a5c-9545-881461cbddf3 None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:40.583906 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 54/101] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 30 15:45:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 15:45:40.586110 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173749e2-fc1f-4e78-8e01-6e90500eb517 None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:40.586394 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173749e2-fc1f-4e78-8e01-6e90500eb517 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:40.586664 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173749e2-fc1f-4e78-8e01-6e90500eb517 None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:40.589873 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 48/102] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 30 15:45:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 15:45:40.591112 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab16e2bf-a8d5-4b05-859f-db0d93da0b90 None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:40.591398 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab16e2bf-a8d5-4b05-859f-db0d93da0b90 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:40.591671 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab16e2bf-a8d5-4b05-859f-db0d93da0b90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:40.594657 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 55/103] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 30 15:45:40 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 15:45:40.621072 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565fc855-06be-47fa-8a27-948e63d47006 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:40.621426 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565fc855-06be-47fa-8a27-948e63d47006 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:40.622030 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565fc855-06be-47fa-8a27-948e63d47006 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:40.647721 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-ab16e2bf-a8d5-4b05-859f-db0d93da0b90 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:45:40.648950 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-ab16e2bf-a8d5-4b05-859f-db0d93da0b90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:45:40.656747 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-565fc855-06be-47fa-8a27-948e63d47006 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:45:40.663039 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-565fc855-06be-47fa-8a27-948e63d47006 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:45:40.743907 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 49/104] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 30 15:45:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 15:45:40.786538 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 56/105] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 30 15:45:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 15:45:40.890189 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab16e2bf-a8d5-4b05-859f-db0d93da0b90 None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:45:40.946188 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:45:41.014966 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab16e2bf-a8d5-4b05-859f-db0d93da0b90 None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:45:41.065005 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0e1dd4-e813-410f-a1f3-51ad1c06c878 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:45:41.065750 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0e1dd4-e813-410f-a1f3-51ad1c06c878 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:41.066032 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0e1dd4-e813-410f-a1f3-51ad1c06c878 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:41.066359 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0e1dd4-e813-410f-a1f3-51ad1c06c878 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:41.082077 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0e1dd4-e813-410f-a1f3-51ad1c06c878 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 15:45:41.087079 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application [None req-1a0e1dd4-e813-410f-a1f3-51ad1c06c878 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 30 15:45:41.087079 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:45:41.087079 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:45:41.087079 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:45:41.087079 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:45:41.087079 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:45:41.087079 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:45:41.087079 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:45:41.087079 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:45:41.087079 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:45:41.087079 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:45:41.087079 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:45:41.087079 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 15:45:41.087079 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 15:45:41.087079 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 15:45:41.087079 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 15:45:41.087927 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 15:45:41.087927 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 15:45:41.087927 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 15:45:41.087927 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 15:45:41.087927 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:45:41.087927 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:45:41.087927 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:45:41.087927 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:45:41.087927 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:45:41.087927 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:45:41.087927 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:45:41.087927 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application with Lock( Aug 30 15:45:41.087927 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:45:41.087927 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:45:41.087927 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:45:41.087927 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:45:41.088718 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:45:41.088718 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:45:41.088718 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:45:41.088718 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:45:41.088718 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 15:45:41.088718 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 15:45:41.088718 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 15:45:41.088718 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 15:45:41.088718 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 15:45:41.088718 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 15:45:41.088718 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 30 15:45:41.088718 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Aug 30 15:45:41.088718 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 50/106] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Aug 30 15:45:40 2022] GET /identity/v3/roles/member => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 15:45:41.095365 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565fc855-06be-47fa-8a27-948e63d47006 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:45:41.103548 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0e1dd4-e813-410f-a1f3-51ad1c06c878 None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:45:41.231402 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0e1dd4-e813-410f-a1f3-51ad1c06c878 None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:45:41.245362 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:45:41.287866 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da00c375-1f29-4bee-851e-3e809c163691 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:45:41.288511 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da00c375-1f29-4bee-851e-3e809c163691 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:41.288790 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da00c375-1f29-4bee-851e-3e809c163691 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:41.289060 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da00c375-1f29-4bee-851e-3e809c163691 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:41.301276 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da00c375-1f29-4bee-851e-3e809c163691 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 15:45:41.305069 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565fc855-06be-47fa-8a27-948e63d47006 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:45:41.306084 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application [None req-da00c375-1f29-4bee-851e-3e809c163691 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 30 15:45:41.306084 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:45:41.306084 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:45:41.306084 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:45:41.306084 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:45:41.306084 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:45:41.306084 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:45:41.306084 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:45:41.306084 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:45:41.306084 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:45:41.306084 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:45:41.306084 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:45:41.306084 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 15:45:41.306084 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 15:45:41.306084 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 15:45:41.306084 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 15:45:41.307084 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 15:45:41.307084 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 15:45:41.307084 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 15:45:41.307084 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 15:45:41.307084 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:45:41.307084 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:45:41.307084 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:45:41.307084 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:45:41.307084 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:45:41.307084 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:45:41.307084 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:45:41.307084 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application with Lock( Aug 30 15:45:41.307084 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:45:41.307084 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:45:41.307084 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:45:41.307084 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:45:41.307847 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:45:41.307847 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:45:41.307847 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:45:41.307847 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:45:41.307847 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 15:45:41.307847 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 15:45:41.307847 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 15:45:41.307847 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 15:45:41.307847 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 15:45:41.307847 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 15:45:41.307847 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 30 15:45:41.307847 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Aug 30 15:45:41.307847 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 51/107] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Aug 30 15:45:41 2022] GET /identity/v3/roles/reader => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 15:45:41.308462 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:45:41.308524 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da00c375-1f29-4bee-851e-3e809c163691 None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:45:41.350637 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90673ebc-09e0-4e8f-9efe-aeb29503ed11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:45:41.351272 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90673ebc-09e0-4e8f-9efe-aeb29503ed11 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:41.351560 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90673ebc-09e0-4e8f-9efe-aeb29503ed11 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:41.351839 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90673ebc-09e0-4e8f-9efe-aeb29503ed11 None admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:41.352897 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90673ebc-09e0-4e8f-9efe-aeb29503ed11 None admin] RBAC: Adding query filter params (name=member) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:45:41.353209 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90673ebc-09e0-4e8f-9efe-aeb29503ed11 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:45:41.355257 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90673ebc-09e0-4e8f-9efe-aeb29503ed11 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:45:41.372355 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 57/108] 10.222.0.20 () {56 vars in 1101 bytes} [Tue Aug 30 15:45:41 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:45:41.373443 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90673ebc-09e0-4e8f-9efe-aeb29503ed11 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:45:41.433863 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da00c375-1f29-4bee-851e-3e809c163691 None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:45:41.472998 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab61a3a8-9240-4b79-9e2b-96c52faf5736 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:45:41.473761 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab61a3a8-9240-4b79-9e2b-96c52faf5736 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:41.474040 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab61a3a8-9240-4b79-9e2b-96c52faf5736 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:41.474313 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab61a3a8-9240-4b79-9e2b-96c52faf5736 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:41.485131 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab61a3a8-9240-4b79-9e2b-96c52faf5736 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 15:45:41.488171 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90673ebc-09e0-4e8f-9efe-aeb29503ed11 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:45:41.490107 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application [None req-ab61a3a8-9240-4b79-9e2b-96c52faf5736 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 15:45:41.490107 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:45:41.490107 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:45:41.490107 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:45:41.490107 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:45:41.490107 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:45:41.490107 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:45:41.490107 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:45:41.490107 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:45:41.490107 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:45:41.490107 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:45:41.490107 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:45:41.490107 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 15:45:41.490107 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 15:45:41.490107 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 15:45:41.490107 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 15:45:41.490961 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 15:45:41.490961 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 15:45:41.490961 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 15:45:41.490961 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 15:45:41.490961 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:45:41.490961 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:45:41.490961 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:45:41.490961 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:45:41.490961 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:45:41.490961 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:45:41.490961 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:45:41.490961 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application with Lock( Aug 30 15:45:41.490961 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:45:41.490961 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:45:41.490961 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:45:41.490961 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:45:41.491736 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:45:41.491736 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:45:41.491736 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:45:41.491736 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:45:41.491736 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 15:45:41.491736 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 15:45:41.491736 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 15:45:41.491736 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 15:45:41.491736 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 15:45:41.491736 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 15:45:41.491736 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 15:45:41.491736 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Aug 30 15:45:41.491736 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 52/109] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Aug 30 15:45:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 15:45:41.492357 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab61a3a8-9240-4b79-9e2b-96c52faf5736 None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:45:41.522617 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:45:41.531219 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f40957-b6a0-43e0-ad81-b2ad6064e579 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:45:41.531883 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f40957-b6a0-43e0-ad81-b2ad6064e579 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:41.532163 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f40957-b6a0-43e0-ad81-b2ad6064e579 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:41.532431 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f40957-b6a0-43e0-ad81-b2ad6064e579 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:41.533608 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f40957-b6a0-43e0-ad81-b2ad6064e579 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:45:41.535092 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f40957-b6a0-43e0-ad81-b2ad6064e579 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:45:41.536700 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 58/110] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 30 15:45:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:45:41.537508 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f40957-b6a0-43e0-ad81-b2ad6064e579 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:45:41.570744 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:45:41.607870 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab61a3a8-9240-4b79-9e2b-96c52faf5736 None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:45:41.647093 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210f2ac1-41fa-4018-91ba-e8211fd2b1a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:45:41.647722 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210f2ac1-41fa-4018-91ba-e8211fd2b1a6 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:41.648007 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210f2ac1-41fa-4018-91ba-e8211fd2b1a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:41.648283 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210f2ac1-41fa-4018-91ba-e8211fd2b1a6 None admin] PATH_INFO: `/v3/roles` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:41.648964 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210f2ac1-41fa-4018-91ba-e8211fd2b1a6 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81512) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:45:41.649295 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210f2ac1-41fa-4018-91ba-e8211fd2b1a6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:45:41.650175 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f40957-b6a0-43e0-ad81-b2ad6064e579 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:45:41.650714 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210f2ac1-41fa-4018-91ba-e8211fd2b1a6 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:45:41.661275 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 53/111] 10.222.0.20 () {56 vars in 1101 bytes} [Tue Aug 30 15:45:41 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:45:41.661275 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c127a46-fe53-4cfe-bf88-4672687d2f97 None None] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:41.661275 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c127a46-fe53-4cfe-bf88-4672687d2f97 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:41.661275 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c127a46-fe53-4cfe-bf88-4672687d2f97 None None] PATH_INFO: `/` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:41.662482 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 54/112] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 30 15:45:41 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 15:45:41.664678 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c127a46-fe53-4cfe-bf88-4672687d2f97 None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:45:41.692346 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9c33b9-8945-4522-beeb-70b5879a6f2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:45:41.693011 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9c33b9-8945-4522-beeb-70b5879a6f2f None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:41.693320 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9c33b9-8945-4522-beeb-70b5879a6f2f None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:41.693588 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9c33b9-8945-4522-beeb-70b5879a6f2f None admin] PATH_INFO: `/v3/users/8249204af55844a180f2d1d09c8b1f59` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:41.694733 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9c33b9-8945-4522-beeb-70b5879a6f2f None admin] RBAC: Authorizing `identity:get_user(user_id=8249204af55844a180f2d1d09c8b1f59)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:45:41.696204 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9c33b9-8945-4522-beeb-70b5879a6f2f None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:45:41.698037 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 59/113] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Aug 30 15:45:41 2022] GET /identity/v3/users/8249204af55844a180f2d1d09c8b1f59 => generated 275 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:45:41.698660 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9c33b9-8945-4522-beeb-70b5879a6f2f None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:45:41.766367 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:45:41.777514 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c127a46-fe53-4cfe-bf88-4672687d2f97 None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:45:41.812096 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c158adbe-dd9c-4a25-9f91-f9f4c479754b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:45:41.812799 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c158adbe-dd9c-4a25-9f91-f9f4c479754b None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:41.813080 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c158adbe-dd9c-4a25-9f91-f9f4c479754b None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:41.813383 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c158adbe-dd9c-4a25-9f91-f9f4c479754b None admin] PATH_INFO: `/v3/roles` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:41.813534 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:45:41.814082 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c158adbe-dd9c-4a25-9f91-f9f4c479754b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81512) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:45:41.814378 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c158adbe-dd9c-4a25-9f91-f9f4c479754b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:45:41.815764 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c158adbe-dd9c-4a25-9f91-f9f4c479754b None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:45:41.817654 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9c33b9-8945-4522-beeb-70b5879a6f2f None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:45:41.825349 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 55/114] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Aug 30 15:45:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:45:41.826090 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c158adbe-dd9c-4a25-9f91-f9f4c479754b None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:45:41.860654 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51876eb8-77c3-4fe7-a1e0-d5a12c800de9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:45:41.861421 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51876eb8-77c3-4fe7-a1e0-d5a12c800de9 None admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:41.861702 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51876eb8-77c3-4fe7-a1e0-d5a12c800de9 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:41.861973 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51876eb8-77c3-4fe7-a1e0-d5a12c800de9 None admin] PATH_INFO: `/v3/projects` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:41.862637 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51876eb8-77c3-4fe7-a1e0-d5a12c800de9 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:45:41.864109 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51876eb8-77c3-4fe7-a1e0-d5a12c800de9 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:45:41.892986 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 60/115] 10.222.0.20 () {60 vars in 1141 bytes} [Tue Aug 30 15:45:41 2022] POST /identity/v3/projects => generated 302 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 15:45:41.902828 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d09331-4d1d-4f24-9b62-de13b1822ede None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:41.903196 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d09331-4d1d-4f24-9b62-de13b1822ede None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:41.903417 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d09331-4d1d-4f24-9b62-de13b1822ede None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:41.923207 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-c6d09331-4d1d-4f24-9b62-de13b1822ede None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:45:41.924475 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-c6d09331-4d1d-4f24-9b62-de13b1822ede None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:45:41.935240 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c158adbe-dd9c-4a25-9f91-f9f4c479754b None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:45:41.975305 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59043457-8141-4b17-8731-ef8f8f8eb998 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:45:41.976161 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59043457-8141-4b17-8731-ef8f8f8eb998 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:41.976441 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59043457-8141-4b17-8731-ef8f8f8eb998 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:41.976714 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59043457-8141-4b17-8731-ef8f8f8eb998 None admin] PATH_INFO: `/v3/users/4f46cc84bf04497e86d8762d42fd9a12` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:41.992828 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59043457-8141-4b17-8731-ef8f8f8eb998 None admin] RBAC: Authorizing `identity:get_user(user_id=4f46cc84bf04497e86d8762d42fd9a12)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:45:41.994402 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59043457-8141-4b17-8731-ef8f8f8eb998 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:45:42.001411 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 61/116] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 30 15:45:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 15:45:42.001411 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d09331-4d1d-4f24-9b62-de13b1822ede None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:45:42.001695 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 56/117] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Aug 30 15:45:41 2022] GET /identity/v3/users/4f46cc84bf04497e86d8762d42fd9a12 => generated 289 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:45:42.001695 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59043457-8141-4b17-8731-ef8f8f8eb998 None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:45:42.015515 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:45:42.103694 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:45:42.125795 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d09331-4d1d-4f24-9b62-de13b1822ede None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:45:42.125975 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59043457-8141-4b17-8731-ef8f8f8eb998 None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:45:42.164285 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dad66cb-398c-42e9-a7a4-da08b7e3906c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:45:42.164946 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dad66cb-398c-42e9-a7a4-da08b7e3906c None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:42.165488 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dad66cb-398c-42e9-a7a4-da08b7e3906c None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:42.165770 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dad66cb-398c-42e9-a7a4-da08b7e3906c None admin] PATH_INFO: `/v3/projects/2a4e4930fe3247d3b04ad23013176d45` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:42.167797 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dad66cb-398c-42e9-a7a4-da08b7e3906c None admin] RBAC: Authorizing `identity:get_project(project_id=2a4e4930fe3247d3b04ad23013176d45)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:45:42.169549 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dad66cb-398c-42e9-a7a4-da08b7e3906c None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:45:42.171376 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46d8dfc-38c6-4b3f-b2e6-e61fa0bdbfc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:45:42.171500 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 62/118] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Aug 30 15:45:41 2022] GET /identity/v3/projects/2a4e4930fe3247d3b04ad23013176d45 => generated 298 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:45:42.172111 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46d8dfc-38c6-4b3f-b2e6-e61fa0bdbfc9 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:42.172453 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46d8dfc-38c6-4b3f-b2e6-e61fa0bdbfc9 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:42.172821 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c4d586-1b99-4ce9-81ff-a8c557d40d36 None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:42.172920 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46d8dfc-38c6-4b3f-b2e6-e61fa0bdbfc9 None admin] PATH_INFO: `/v3/users/8b9160d3cbae47a48dc9dee882da999b` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:42.173116 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c4d586-1b99-4ce9-81ff-a8c557d40d36 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:42.173429 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c4d586-1b99-4ce9-81ff-a8c557d40d36 None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:42.174239 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46d8dfc-38c6-4b3f-b2e6-e61fa0bdbfc9 None admin] RBAC: Authorizing `identity:get_user(user_id=8b9160d3cbae47a48dc9dee882da999b)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:45:42.174734 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 63/119] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 30 15:45:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 15:45:42.175366 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c4d586-1b99-4ce9-81ff-a8c557d40d36 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:45:42.179761 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46d8dfc-38c6-4b3f-b2e6-e61fa0bdbfc9 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:45:42.181570 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 57/120] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Aug 30 15:45:41 2022] GET /identity/v3/users/8b9160d3cbae47a48dc9dee882da999b => generated 247 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:45:42.182381 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46d8dfc-38c6-4b3f-b2e6-e61fa0bdbfc9 None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:45:42.267648 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:45:42.293188 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46d8dfc-38c6-4b3f-b2e6-e61fa0bdbfc9 None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:45:42.307088 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c4d586-1b99-4ce9-81ff-a8c557d40d36 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:45:42.330389 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64190463-017f-45d1-98d3-8730d1913a18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:45:42.331124 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64190463-017f-45d1-98d3-8730d1913a18 None admin] REQUEST_METHOD: `PUT` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:42.331328 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64190463-017f-45d1-98d3-8730d1913a18 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:42.331597 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64190463-017f-45d1-98d3-8730d1913a18 None admin] PATH_INFO: `/v3/projects/2a4e4930fe3247d3b04ad23013176d45/users/8249204af55844a180f2d1d09c8b1f59/roles/df8f9f844b1b439181d75a07c1a93418` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:42.333591 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64190463-017f-45d1-98d3-8730d1913a18 None admin] RBAC: Authorizing `identity:create_grant(project_id=2a4e4930fe3247d3b04ad23013176d45, user_id=8249204af55844a180f2d1d09c8b1f59, role_id=df8f9f844b1b439181d75a07c1a93418)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:45:42.335010 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64190463-017f-45d1-98d3-8730d1913a18 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:45:42.343285 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e19743d-b518-49b3-b50b-a7555f7d0ccd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:45:42.343933 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e19743d-b518-49b3-b50b-a7555f7d0ccd None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:42.344214 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e19743d-b518-49b3-b50b-a7555f7d0ccd None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:42.344491 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e19743d-b518-49b3-b50b-a7555f7d0ccd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:42.357338 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 58/121] 10.222.0.20 () {58 vars in 1439 bytes} [Tue Aug 30 15:45:42 2022] PUT /identity/v3/projects/2a4e4930fe3247d3b04ad23013176d45/users/8249204af55844a180f2d1d09c8b1f59/roles/df8f9f844b1b439181d75a07c1a93418 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:45:42.357338 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70628691-42c5-4e93-a5ef-fccecf1471a6 None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:42.357338 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70628691-42c5-4e93-a5ef-fccecf1471a6 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:42.357338 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70628691-42c5-4e93-a5ef-fccecf1471a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:42.372047 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:45:42.376976 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-70628691-42c5-4e93-a5ef-fccecf1471a6 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:45:42.378888 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-70628691-42c5-4e93-a5ef-fccecf1471a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:45:42.431823 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e19743d-b518-49b3-b50b-a7555f7d0ccd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:45:42.433453 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e19743d-b518-49b3-b50b-a7555f7d0ccd None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:45:42.471274 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 59/122] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 30 15:45:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 15:45:42.472180 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70628691-42c5-4e93-a5ef-fccecf1471a6 None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:45:42.582093 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 64/123] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Aug 30 15:45:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 15:45:42.583146 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e19743d-b518-49b3-b50b-a7555f7d0ccd None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:45:42.583905 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:45:42.622922 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70628691-42c5-4e93-a5ef-fccecf1471a6 None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:45:42.648710 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:45:42.686447 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0e4f78-0957-4045-8672-dd8471b7d5dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:45:42.687098 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0e4f78-0957-4045-8672-dd8471b7d5dc None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:42.687437 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0e4f78-0957-4045-8672-dd8471b7d5dc None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:42.687772 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0e4f78-0957-4045-8672-dd8471b7d5dc None admin] PATH_INFO: `/v3/projects/2a4e4930fe3247d3b04ad23013176d45` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:42.689049 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0e4f78-0957-4045-8672-dd8471b7d5dc None admin] RBAC: Authorizing `identity:get_project(project_id=2a4e4930fe3247d3b04ad23013176d45)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:45:42.690799 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0e4f78-0957-4045-8672-dd8471b7d5dc None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:45:42.692493 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 60/124] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Aug 30 15:45:42 2022] GET /identity/v3/projects/2a4e4930fe3247d3b04ad23013176d45 => generated 298 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:45:42.694869 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0e4f78-0957-4045-8672-dd8471b7d5dc None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:45:42.731850 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e19743d-b518-49b3-b50b-a7555f7d0ccd None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:45:42.782370 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9358526-91d4-41c4-a14a-f5ef8e50165b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:45:42.783007 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9358526-91d4-41c4-a14a-f5ef8e50165b None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:42.783272 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9358526-91d4-41c4-a14a-f5ef8e50165b None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:42.783540 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9358526-91d4-41c4-a14a-f5ef8e50165b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:42.794152 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9358526-91d4-41c4-a14a-f5ef8e50165b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 15:45:42.799169 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-e9358526-91d4-41c4-a14a-f5ef8e50165b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 15:45:42.799169 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:45:42.799169 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:45:42.799169 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:45:42.799169 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:45:42.799169 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:45:42.799169 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:45:42.799169 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:45:42.799169 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:45:42.799169 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:45:42.799169 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:45:42.799169 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:45:42.799169 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 15:45:42.799169 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 15:45:42.799169 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 15:45:42.799169 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 15:45:42.800042 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 15:45:42.800042 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 15:45:42.800042 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 15:45:42.800042 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 15:45:42.800042 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:45:42.800042 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:45:42.800042 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:45:42.800042 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:45:42.800042 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:45:42.800042 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:45:42.800042 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:45:42.800042 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Aug 30 15:45:42.800042 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:45:42.800042 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:45:42.800042 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:45:42.800042 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:45:42.800811 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:45:42.800811 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:45:42.800811 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:45:42.800811 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:45:42.800811 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 15:45:42.800811 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 15:45:42.800811 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 15:45:42.800811 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 15:45:42.800811 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 15:45:42.800811 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 15:45:42.800811 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 15:45:42.800811 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Aug 30 15:45:42.800811 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 65/125] 10.222.0.20 () {56 vars in 1114 bytes} [Tue Aug 30 15:45:42 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 15:45:42.801481 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9358526-91d4-41c4-a14a-f5ef8e50165b None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:45:42.844344 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0e4f78-0957-4045-8672-dd8471b7d5dc None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:45:42.880383 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86553304-ef14-46c5-989f-8f502e8b0fd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:45:42.881082 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86553304-ef14-46c5-989f-8f502e8b0fd1 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:42.881432 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86553304-ef14-46c5-989f-8f502e8b0fd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:42.881705 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86553304-ef14-46c5-989f-8f502e8b0fd1 None admin] PATH_INFO: `/v3/projects/2a4e4930fe3247d3b04ad23013176d45` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:42.882867 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86553304-ef14-46c5-989f-8f502e8b0fd1 None admin] RBAC: Authorizing `identity:get_project(project_id=2a4e4930fe3247d3b04ad23013176d45)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:45:42.884336 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86553304-ef14-46c5-989f-8f502e8b0fd1 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:45:42.886330 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 61/126] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Aug 30 15:45:42 2022] GET /identity/v3/projects/2a4e4930fe3247d3b04ad23013176d45 => generated 298 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:45:42.887107 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86553304-ef14-46c5-989f-8f502e8b0fd1 None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:45:42.891787 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:45:42.923558 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9358526-91d4-41c4-a14a-f5ef8e50165b None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:45:42.937799 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:45:42.973743 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7dedea9-0a8f-4954-bc58-8ed83d146ddf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:45:42.974431 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dedea9-0a8f-4954-bc58-8ed83d146ddf None admin] REQUEST_METHOD: `PUT` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:42.974733 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dedea9-0a8f-4954-bc58-8ed83d146ddf None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:42.975229 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dedea9-0a8f-4954-bc58-8ed83d146ddf None admin] PATH_INFO: `/v3/projects/2a4e4930fe3247d3b04ad23013176d45/users/8b9160d3cbae47a48dc9dee882da999b/roles/f265386950a641ac8413b53eda4ae34d` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:42.978090 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7dedea9-0a8f-4954-bc58-8ed83d146ddf None admin] RBAC: Authorizing `identity:create_grant(project_id=2a4e4930fe3247d3b04ad23013176d45, user_id=8b9160d3cbae47a48dc9dee882da999b, role_id=f265386950a641ac8413b53eda4ae34d)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:45:42.980814 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7dedea9-0a8f-4954-bc58-8ed83d146ddf None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:45:43.001018 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86553304-ef14-46c5-989f-8f502e8b0fd1 None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:45:43.007165 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 66/127] 10.222.0.20 () {58 vars in 1439 bytes} [Tue Aug 30 15:45:42 2022] PUT /identity/v3/projects/2a4e4930fe3247d3b04ad23013176d45/users/8b9160d3cbae47a48dc9dee882da999b/roles/f265386950a641ac8413b53eda4ae34d => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:45:43.008519 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7dedea9-0a8f-4954-bc58-8ed83d146ddf None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:45:43.047434 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cff799-65b1-4959-94b0-9169278db8eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:45:43.048464 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cff799-65b1-4959-94b0-9169278db8eb None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:43.048942 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cff799-65b1-4959-94b0-9169278db8eb None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:43.049466 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cff799-65b1-4959-94b0-9169278db8eb None admin] PATH_INFO: `/v3/roles/member` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:43.061999 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4cff799-65b1-4959-94b0-9169278db8eb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 15:45:43.067266 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application [None req-e4cff799-65b1-4959-94b0-9169278db8eb None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 30 15:45:43.067266 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:45:43.067266 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:45:43.067266 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:45:43.067266 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:45:43.067266 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:45:43.067266 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:45:43.067266 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:45:43.067266 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:45:43.067266 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:45:43.067266 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:45:43.067266 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:45:43.067266 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 15:45:43.067266 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 15:45:43.067266 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 15:45:43.067266 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 15:45:43.068116 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 15:45:43.068116 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 15:45:43.068116 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 15:45:43.068116 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 15:45:43.068116 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:45:43.068116 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:45:43.068116 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:45:43.068116 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:45:43.068116 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:45:43.068116 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:45:43.068116 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:45:43.068116 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application with Lock( Aug 30 15:45:43.068116 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:45:43.068116 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:45:43.068116 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:45:43.068116 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:45:43.068898 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:45:43.068898 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:45:43.068898 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:45:43.068898 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:45:43.068898 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 15:45:43.068898 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 15:45:43.068898 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 15:45:43.068898 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 15:45:43.068898 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 15:45:43.068898 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 15:45:43.068898 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 30 15:45:43.068898 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Aug 30 15:45:43.071204 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 62/128] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Aug 30 15:45:42 2022] GET /identity/v3/roles/member => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 15:45:43.072291 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cff799-65b1-4959-94b0-9169278db8eb None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:45:43.141152 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7dedea9-0a8f-4954-bc58-8ed83d146ddf None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:45:43.164117 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:45:43.178789 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b887f100-df30-42e4-86e9-7f9a68c06fe6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:45:43.179769 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b887f100-df30-42e4-86e9-7f9a68c06fe6 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:43.180244 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b887f100-df30-42e4-86e9-7f9a68c06fe6 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:43.180706 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b887f100-df30-42e4-86e9-7f9a68c06fe6 None admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:43.181926 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b887f100-df30-42e4-86e9-7f9a68c06fe6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:45:43.182427 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b887f100-df30-42e4-86e9-7f9a68c06fe6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:45:43.184189 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b887f100-df30-42e4-86e9-7f9a68c06fe6 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:45:43.193722 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 67/129] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Aug 30 15:45:43 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:45:43.195016 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b887f100-df30-42e4-86e9-7f9a68c06fe6 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:45:43.212519 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cff799-65b1-4959-94b0-9169278db8eb None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:45:43.231143 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:45:43.286530 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3640a72-fb7d-4c28-81df-e2a0ce6b80b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:45:43.287176 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3640a72-fb7d-4c28-81df-e2a0ce6b80b8 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:43.287743 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3640a72-fb7d-4c28-81df-e2a0ce6b80b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:43.287848 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3640a72-fb7d-4c28-81df-e2a0ce6b80b8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:43.288556 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3640a72-fb7d-4c28-81df-e2a0ce6b80b8 None admin] RBAC: Adding query filter params (role.id=5d293e67c8e747e6910e95cf65e56633, scope.project.id=2a4e4930fe3247d3b04ad23013176d45, user.id=4f46cc84bf04497e86d8762d42fd9a12) {{(pid=81512) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:45:43.288850 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3640a72-fb7d-4c28-81df-e2a0ce6b80b8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:45:43.290414 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3640a72-fb7d-4c28-81df-e2a0ce6b80b8 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:45:43.298073 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 63/130] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Aug 30 15:45:43 2022] GET /identity/v3/role_assignments?user.id=4f46cc84bf04497e86d8762d42fd9a12&scope.project.id=2a4e4930fe3247d3b04ad23013176d45&role.id=5d293e67c8e747e6910e95cf65e56633 => generated 259 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:45:43.298512 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3640a72-fb7d-4c28-81df-e2a0ce6b80b8 None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:45:43.331645 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b887f100-df30-42e4-86e9-7f9a68c06fe6 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:45:43.375944 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095d895f-0c8a-4503-ad6a-5cbce8d3ad65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:45:43.376976 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095d895f-0c8a-4503-ad6a-5cbce8d3ad65 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:43.377482 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095d895f-0c8a-4503-ad6a-5cbce8d3ad65 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:43.377980 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095d895f-0c8a-4503-ad6a-5cbce8d3ad65 None admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:43.378868 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095d895f-0c8a-4503-ad6a-5cbce8d3ad65 None admin] RBAC: Adding query filter params (name=member) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:45:43.379373 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095d895f-0c8a-4503-ad6a-5cbce8d3ad65 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:45:43.381094 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095d895f-0c8a-4503-ad6a-5cbce8d3ad65 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:45:43.397753 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 68/131] 10.222.0.20 () {56 vars in 1101 bytes} [Tue Aug 30 15:45:43 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:45:43.402917 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095d895f-0c8a-4503-ad6a-5cbce8d3ad65 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:45:43.412437 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3640a72-fb7d-4c28-81df-e2a0ce6b80b8 None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:45:43.446694 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:45:43.466068 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0bb092-1f56-42cb-b1cc-a72f3fa24c45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:45:43.466745 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0bb092-1f56-42cb-b1cc-a72f3fa24c45 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:43.467150 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0bb092-1f56-42cb-b1cc-a72f3fa24c45 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:43.467431 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0bb092-1f56-42cb-b1cc-a72f3fa24c45 None admin] PATH_INFO: `/v3/users/8249204af55844a180f2d1d09c8b1f59` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:43.468765 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0bb092-1f56-42cb-b1cc-a72f3fa24c45 None admin] RBAC: Authorizing `identity:get_user(user_id=8249204af55844a180f2d1d09c8b1f59)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:45:43.470393 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0bb092-1f56-42cb-b1cc-a72f3fa24c45 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:45:43.472181 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 64/132] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Aug 30 15:45:43 2022] GET /identity/v3/users/8249204af55844a180f2d1d09c8b1f59 => generated 275 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:45:43.483598 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0bb092-1f56-42cb-b1cc-a72f3fa24c45 None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:45:43.592568 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:45:43.599096 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0bb092-1f56-42cb-b1cc-a72f3fa24c45 None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:45:43.611456 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095d895f-0c8a-4503-ad6a-5cbce8d3ad65 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:45:43.639541 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b787089a-3f2e-4cbf-b4c5-c818cdda6ddd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:45:43.640351 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b787089a-3f2e-4cbf-b4c5-c818cdda6ddd None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:43.640586 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b787089a-3f2e-4cbf-b4c5-c818cdda6ddd None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:43.640817 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b787089a-3f2e-4cbf-b4c5-c818cdda6ddd None admin] PATH_INFO: `/v3/projects/2a4e4930fe3247d3b04ad23013176d45` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:43.642022 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b787089a-3f2e-4cbf-b4c5-c818cdda6ddd None admin] RBAC: Authorizing `identity:get_project(project_id=2a4e4930fe3247d3b04ad23013176d45)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:45:43.643512 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b787089a-3f2e-4cbf-b4c5-c818cdda6ddd None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:45:43.645210 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 65/133] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Aug 30 15:45:43 2022] GET /identity/v3/projects/2a4e4930fe3247d3b04ad23013176d45 => generated 298 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 15:45:43.652497 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b787089a-3f2e-4cbf-b4c5-c818cdda6ddd None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:45:43.727300 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:45:43.746512 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ea3bcf-4dec-40a6-8e79-25541f469dbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:45:43.747514 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ea3bcf-4dec-40a6-8e79-25541f469dbc None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:43.747793 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ea3bcf-4dec-40a6-8e79-25541f469dbc None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:43.748066 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ea3bcf-4dec-40a6-8e79-25541f469dbc None admin] PATH_INFO: `/v3/users/8249204af55844a180f2d1d09c8b1f59` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:43.749241 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ea3bcf-4dec-40a6-8e79-25541f469dbc None admin] RBAC: Authorizing `identity:get_user(user_id=8249204af55844a180f2d1d09c8b1f59)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:45:43.751126 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ea3bcf-4dec-40a6-8e79-25541f469dbc None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:45:43.753034 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 69/134] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Aug 30 15:45:43 2022] GET /identity/v3/users/8249204af55844a180f2d1d09c8b1f59 => generated 275 bytes in 350 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 15:45:43.758618 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ea3bcf-4dec-40a6-8e79-25541f469dbc None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:45:43.799205 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b787089a-3f2e-4cbf-b4c5-c818cdda6ddd None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:45:43.855360 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fc3dde-9fdb-49b5-b9b1-2e477a4fe02a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:45:43.856023 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fc3dde-9fdb-49b5-b9b1-2e477a4fe02a None admin] REQUEST_METHOD: `PUT` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:43.856303 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fc3dde-9fdb-49b5-b9b1-2e477a4fe02a None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:43.856576 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fc3dde-9fdb-49b5-b9b1-2e477a4fe02a None admin] PATH_INFO: `/v3/projects/2a4e4930fe3247d3b04ad23013176d45/users/8249204af55844a180f2d1d09c8b1f59/roles/6b4385c3b449456d848c9abb16766d25` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:43.863049 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fc3dde-9fdb-49b5-b9b1-2e477a4fe02a None admin] RBAC: Authorizing `identity:create_grant(project_id=2a4e4930fe3247d3b04ad23013176d45, user_id=8249204af55844a180f2d1d09c8b1f59, role_id=6b4385c3b449456d848c9abb16766d25)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:45:43.864527 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fc3dde-9fdb-49b5-b9b1-2e477a4fe02a None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:45:43.894979 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 66/135] 10.222.0.20 () {58 vars in 1439 bytes} [Tue Aug 30 15:45:43 2022] PUT /identity/v3/projects/2a4e4930fe3247d3b04ad23013176d45/users/8249204af55844a180f2d1d09c8b1f59/roles/6b4385c3b449456d848c9abb16766d25 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:45:43.947530 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ea3bcf-4dec-40a6-8e79-25541f469dbc None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:45:43.989560 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1ec7af-8a89-4ea5-ae8b-af91c8062360 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:45:43.990877 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1ec7af-8a89-4ea5-ae8b-af91c8062360 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:43.991358 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1ec7af-8a89-4ea5-ae8b-af91c8062360 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:43.991874 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1ec7af-8a89-4ea5-ae8b-af91c8062360 None admin] PATH_INFO: `/v3/projects/8ed1a9892a9a428a92a900c9ce92f2b0` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:43.993407 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1ec7af-8a89-4ea5-ae8b-af91c8062360 None admin] RBAC: Authorizing `identity:get_project(project_id=8ed1a9892a9a428a92a900c9ce92f2b0)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:45:43.995241 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1ec7af-8a89-4ea5-ae8b-af91c8062360 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:45:43.997574 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 70/136] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Aug 30 15:45:43 2022] GET /identity/v3/projects/8ed1a9892a9a428a92a900c9ce92f2b0 => generated 312 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 15:45:44.005285 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fc3dde-9fdb-49b5-b9b1-2e477a4fe02a None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:45:44.092572 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb4225b-a69b-477e-bb95-cce2b34e7e4d None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:44.093206 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb4225b-a69b-477e-bb95-cce2b34e7e4d None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:44.093778 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb4225b-a69b-477e-bb95-cce2b34e7e4d None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:44.095413 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 71/137] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 30 15:45:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 15:45:44.111357 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a0a053-d521-4f19-89b2-27e4dd943f7a None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:44.111974 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a0a053-d521-4f19-89b2-27e4dd943f7a None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:44.112481 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a0a053-d521-4f19-89b2-27e4dd943f7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:44.149098 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-23a0a053-d521-4f19-89b2-27e4dd943f7a None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:45:44.150830 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-23a0a053-d521-4f19-89b2-27e4dd943f7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:45:44.159807 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fc3dde-9fdb-49b5-b9b1-2e477a4fe02a None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:45:44.201737 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:45:44.203260 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:45:44.215427 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688ba42d-b816-425f-8e9e-bbde59f5ddab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:45:44.216495 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688ba42d-b816-425f-8e9e-bbde59f5ddab None admin] REQUEST_METHOD: `PUT` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:44.216949 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688ba42d-b816-425f-8e9e-bbde59f5ddab None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:44.217415 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688ba42d-b816-425f-8e9e-bbde59f5ddab None admin] PATH_INFO: `/v3/projects/8ed1a9892a9a428a92a900c9ce92f2b0/users/8249204af55844a180f2d1d09c8b1f59/roles/df8f9f844b1b439181d75a07c1a93418` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:44.220443 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688ba42d-b816-425f-8e9e-bbde59f5ddab None admin] RBAC: Authorizing `identity:create_grant(project_id=8ed1a9892a9a428a92a900c9ce92f2b0, user_id=8249204af55844a180f2d1d09c8b1f59, role_id=df8f9f844b1b439181d75a07c1a93418)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:45:44.222099 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688ba42d-b816-425f-8e9e-bbde59f5ddab None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:45:44.239236 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 67/138] 10.222.0.20 () {58 vars in 1439 bytes} [Tue Aug 30 15:45:44 2022] PUT /identity/v3/projects/8ed1a9892a9a428a92a900c9ce92f2b0/users/8249204af55844a180f2d1d09c8b1f59/roles/df8f9f844b1b439181d75a07c1a93418 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:45:44.263398 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 72/139] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 30 15:45:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 15:45:44.664336 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688ba42d-b816-425f-8e9e-bbde59f5ddab None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:45:44.767835 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688ba42d-b816-425f-8e9e-bbde59f5ddab None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:45:44.804425 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45af6144-303b-4d3c-b19a-e24651b233ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:45:44.805071 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45af6144-303b-4d3c-b19a-e24651b233ad None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:44.805380 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45af6144-303b-4d3c-b19a-e24651b233ad None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:44.805650 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45af6144-303b-4d3c-b19a-e24651b233ad None admin] PATH_INFO: `/v3/domains/default` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:44.806776 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45af6144-303b-4d3c-b19a-e24651b233ad None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:45:44.808418 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45af6144-303b-4d3c-b19a-e24651b233ad None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:45:44.810046 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 68/140] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 30 15:45:44 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:45:44.815721 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a0a053-d521-4f19-89b2-27e4dd943f7a None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:45:44.954337 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a0a053-d521-4f19-89b2-27e4dd943f7a None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:45:45.004441 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a035cdf5-1b72-4b4b-b739-95d1a7980f7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:45:45.005432 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a035cdf5-1b72-4b4b-b739-95d1a7980f7d None admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:45.005887 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a035cdf5-1b72-4b4b-b739-95d1a7980f7d None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:45.006331 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a035cdf5-1b72-4b4b-b739-95d1a7980f7d None admin] PATH_INFO: `/v3/users` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:45.007205 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a035cdf5-1b72-4b4b-b739-95d1a7980f7d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:45:45.009024 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a035cdf5-1b72-4b4b-b739-95d1a7980f7d None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:45:45.053899 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 73/141] 10.222.0.20 () {60 vars in 1132 bytes} [Tue Aug 30 15:45:44 2022] POST /identity/v3/users => generated 283 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 15:45:45.123672 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead7f1d7-9ec5-45cd-bae8-6aee77144a3f None None] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:45.123959 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead7f1d7-9ec5-45cd-bae8-6aee77144a3f None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:45.124250 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead7f1d7-9ec5-45cd-bae8-6aee77144a3f None None] PATH_INFO: `/` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:45.125878 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 69/142] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 30 15:45:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 15:45:45.139509 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6e0390-91cd-4a93-a318-d5d9e3ba72b6 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:45.140081 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6e0390-91cd-4a93-a318-d5d9e3ba72b6 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:45.140580 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6e0390-91cd-4a93-a318-d5d9e3ba72b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:45.164890 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-4a6e0390-91cd-4a93-a318-d5d9e3ba72b6 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:45:45.166547 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-4a6e0390-91cd-4a93-a318-d5d9e3ba72b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:45:45.179123 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:45:45.254677 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 74/143] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 30 15:45:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 15:45:45.610473 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead7f1d7-9ec5-45cd-bae8-6aee77144a3f None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:45:45.679744 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:45:45.747038 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead7f1d7-9ec5-45cd-bae8-6aee77144a3f None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:45:45.795699 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c53e641-0dd6-416d-90cb-3474bb85aaa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:45:45.796529 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c53e641-0dd6-416d-90cb-3474bb85aaa1 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:45.796893 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c53e641-0dd6-416d-90cb-3474bb85aaa1 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:45.797168 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c53e641-0dd6-416d-90cb-3474bb85aaa1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:45.811397 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c53e641-0dd6-416d-90cb-3474bb85aaa1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 15:45:45.816449 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application [None req-4c53e641-0dd6-416d-90cb-3474bb85aaa1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 30 15:45:45.816449 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:45:45.816449 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:45:45.816449 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:45:45.816449 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:45:45.816449 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:45:45.816449 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:45:45.816449 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:45:45.816449 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:45:45.816449 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:45:45.816449 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:45:45.816449 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:45:45.816449 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 15:45:45.816449 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 15:45:45.816449 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 15:45:45.816449 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 15:45:45.817296 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 15:45:45.817296 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 15:45:45.817296 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 15:45:45.817296 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 15:45:45.817296 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:45:45.817296 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:45:45.817296 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:45:45.817296 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:45:45.817296 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:45:45.817296 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:45:45.817296 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:45:45.817296 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application with Lock( Aug 30 15:45:45.817296 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:45:45.817296 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:45:45.817296 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:45:45.817296 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:45:45.818213 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:45:45.818213 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:45:45.818213 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:45:45.818213 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:45:45.818213 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 15:45:45.818213 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 15:45:45.818213 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 15:45:45.818213 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 15:45:45.818213 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 15:45:45.818213 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 15:45:45.818213 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 30 15:45:45.818213 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Aug 30 15:45:45.818213 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 70/144] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Aug 30 15:45:45 2022] GET /identity/v3/roles/member => generated 84 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 15:45:45.826818 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6e0390-91cd-4a93-a318-d5d9e3ba72b6 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:45:45.960265 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6e0390-91cd-4a93-a318-d5d9e3ba72b6 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:45:45.999456 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30967122-39e6-4721-8c38-79e749e23384 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:45:46.000148 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30967122-39e6-4721-8c38-79e749e23384 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:46.000497 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30967122-39e6-4721-8c38-79e749e23384 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:46.000775 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30967122-39e6-4721-8c38-79e749e23384 None admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:46.001498 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30967122-39e6-4721-8c38-79e749e23384 None admin] RBAC: Adding query filter params (name=member) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:45:46.001806 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30967122-39e6-4721-8c38-79e749e23384 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:45:46.003230 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30967122-39e6-4721-8c38-79e749e23384 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:45:46.012216 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 75/145] 10.222.0.20 () {56 vars in 1101 bytes} [Tue Aug 30 15:45:45 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:45:46.017748 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c53e641-0dd6-416d-90cb-3474bb85aaa1 None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:45:46.133895 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c53e641-0dd6-416d-90cb-3474bb85aaa1 None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:45:46.150386 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:45:46.173522 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30aac222-f8f6-4914-8dd2-1d9fcf25bd05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:45:46.174164 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30aac222-f8f6-4914-8dd2-1d9fcf25bd05 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:46.174445 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30aac222-f8f6-4914-8dd2-1d9fcf25bd05 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:46.174720 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30aac222-f8f6-4914-8dd2-1d9fcf25bd05 None admin] PATH_INFO: `/v3/users/8249204af55844a180f2d1d09c8b1f59` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:46.175930 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30aac222-f8f6-4914-8dd2-1d9fcf25bd05 None admin] RBAC: Authorizing `identity:get_user(user_id=8249204af55844a180f2d1d09c8b1f59)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:45:46.177488 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30aac222-f8f6-4914-8dd2-1d9fcf25bd05 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:45:46.179279 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 71/146] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Aug 30 15:45:46 2022] GET /identity/v3/users/8249204af55844a180f2d1d09c8b1f59 => generated 275 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:45:46.185001 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30967122-39e6-4721-8c38-79e749e23384 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:45:46.197012 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:45:46.232929 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764249b3-93cc-4ba6-ace3-1d0dbaa4cf40 None None] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:46.233111 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764249b3-93cc-4ba6-ace3-1d0dbaa4cf40 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:46.233431 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764249b3-93cc-4ba6-ace3-1d0dbaa4cf40 None None] PATH_INFO: `/` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:46.235025 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 72/147] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 30 15:45:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 15:45:46.259534 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246b7d6e-996f-4d76-9d4c-838d5ed74dcf None None] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:46.259827 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246b7d6e-996f-4d76-9d4c-838d5ed74dcf None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:46.260094 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246b7d6e-996f-4d76-9d4c-838d5ed74dcf None None] PATH_INFO: `/` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:46.262041 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 73/148] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 30 15:45:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 15:45:46.264011 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72100fb-cdce-4fd2-8f4d-21a336be63cf None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:46.264295 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72100fb-cdce-4fd2-8f4d-21a336be63cf None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:46.264570 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72100fb-cdce-4fd2-8f4d-21a336be63cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:46.290031 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-d72100fb-cdce-4fd2-8f4d-21a336be63cf None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:45:46.291578 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-d72100fb-cdce-4fd2-8f4d-21a336be63cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:45:46.308344 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30967122-39e6-4721-8c38-79e749e23384 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:45:46.370059 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbe19f4-119f-4d5a-8b12-3713b371661f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:45:46.371261 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbe19f4-119f-4d5a-8b12-3713b371661f None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:46.371261 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbe19f4-119f-4d5a-8b12-3713b371661f None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:46.371261 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbe19f4-119f-4d5a-8b12-3713b371661f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:46.439247 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 74/149] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 30 15:45:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 15:45:46.440632 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f7ceab-3acb-41aa-861e-c8bcf4d94b7d None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:46.440923 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f7ceab-3acb-41aa-861e-c8bcf4d94b7d None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:46.441186 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f7ceab-3acb-41aa-861e-c8bcf4d94b7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:46.445909 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbe19f4-119f-4d5a-8b12-3713b371661f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:45:46.447403 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbe19f4-119f-4d5a-8b12-3713b371661f None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:45:46.458145 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:45:46.479558 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-76f7ceab-3acb-41aa-861e-c8bcf4d94b7d None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:45:46.480758 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-76f7ceab-3acb-41aa-861e-c8bcf4d94b7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:45:46.555279 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 76/150] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Aug 30 15:45:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 15:45:46.560719 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbe19f4-119f-4d5a-8b12-3713b371661f None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:45:46.594637 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 75/151] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 30 15:45:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 15:45:46.675425 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbe19f4-119f-4d5a-8b12-3713b371661f None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:45:46.687279 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:45:46.711536 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba151c4f-7f90-4bfe-880e-a0af43a2ebe6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:45:46.712725 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba151c4f-7f90-4bfe-880e-a0af43a2ebe6 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:46.713274 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba151c4f-7f90-4bfe-880e-a0af43a2ebe6 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:46.713720 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba151c4f-7f90-4bfe-880e-a0af43a2ebe6 None admin] PATH_INFO: `/v3/projects/2a4e4930fe3247d3b04ad23013176d45` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:46.715071 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba151c4f-7f90-4bfe-880e-a0af43a2ebe6 None admin] RBAC: Authorizing `identity:get_project(project_id=2a4e4930fe3247d3b04ad23013176d45)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:45:46.716709 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba151c4f-7f90-4bfe-880e-a0af43a2ebe6 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:45:46.718966 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 77/152] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Aug 30 15:45:46 2022] GET /identity/v3/projects/2a4e4930fe3247d3b04ad23013176d45 => generated 298 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:45:46.725266 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f7ceab-3acb-41aa-861e-c8bcf4d94b7d None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:45:46.791074 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:45:46.856807 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f7ceab-3acb-41aa-861e-c8bcf4d94b7d None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:45:46.897325 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcb7908-4ff7-4582-83f4-139d70993148 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:45:46.897958 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcb7908-4ff7-4582-83f4-139d70993148 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:46.898236 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcb7908-4ff7-4582-83f4-139d70993148 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:46.898512 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcb7908-4ff7-4582-83f4-139d70993148 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:46.899254 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bcb7908-4ff7-4582-83f4-139d70993148 None admin] RBAC: Adding query filter params (role.id=df8f9f844b1b439181d75a07c1a93418, scope.project.id=2a4e4930fe3247d3b04ad23013176d45, user.id=8249204af55844a180f2d1d09c8b1f59) {{(pid=81512) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:45:46.899563 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bcb7908-4ff7-4582-83f4-139d70993148 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:45:46.901031 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bcb7908-4ff7-4582-83f4-139d70993148 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:45:46.907634 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 76/153] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Aug 30 15:45:46 2022] GET /identity/v3/role_assignments?user.id=8249204af55844a180f2d1d09c8b1f59&scope.project.id=2a4e4930fe3247d3b04ad23013176d45&role.id=df8f9f844b1b439181d75a07c1a93418 => generated 608 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:45:47.077106 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba151c4f-7f90-4bfe-880e-a0af43a2ebe6 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:45:47.147355 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcb7908-4ff7-4582-83f4-139d70993148 None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:45:47.196192 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba151c4f-7f90-4bfe-880e-a0af43a2ebe6 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:45:47.248795 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9765ed-74fb-4151-a01b-cd9647d0abec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:45:47.249879 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9765ed-74fb-4151-a01b-cd9647d0abec None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:47.250351 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9765ed-74fb-4151-a01b-cd9647d0abec None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:47.250907 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9765ed-74fb-4151-a01b-cd9647d0abec None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:47.273932 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9765ed-74fb-4151-a01b-cd9647d0abec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 15:45:47.286117 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-1c9765ed-74fb-4151-a01b-cd9647d0abec None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 30 15:45:47.286117 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:45:47.286117 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:45:47.286117 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:45:47.286117 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:45:47.286117 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:45:47.286117 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:45:47.286117 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:45:47.286117 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:45:47.286117 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:45:47.286117 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:45:47.286117 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:45:47.286117 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 15:45:47.286117 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 15:45:47.286117 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 15:45:47.286117 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 15:45:47.287100 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 15:45:47.287100 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 15:45:47.287100 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 15:45:47.287100 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 15:45:47.287100 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:45:47.287100 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:45:47.287100 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:45:47.287100 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:45:47.287100 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:45:47.287100 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:45:47.287100 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:45:47.287100 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Aug 30 15:45:47.287100 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:45:47.287100 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:45:47.287100 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:45:47.287100 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:45:47.287906 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:45:47.287906 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:45:47.287906 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:45:47.287906 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:45:47.287906 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 15:45:47.287906 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 15:45:47.287906 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 15:45:47.287906 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 15:45:47.287906 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 15:45:47.287906 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 15:45:47.287906 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 30 15:45:47.287906 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Aug 30 15:45:47.290154 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 78/154] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Aug 30 15:45:47 2022] GET /identity/v3/roles/reader => generated 84 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 15:45:47.292240 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd21d253-fa53-47f7-9b38-fbb1f92d1ac7 None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:47.292690 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd21d253-fa53-47f7-9b38-fbb1f92d1ac7 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:47.293122 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd21d253-fa53-47f7-9b38-fbb1f92d1ac7 None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:47.294578 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 79/155] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 30 15:45:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 15:45:47.295810 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd21d253-fa53-47f7-9b38-fbb1f92d1ac7 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:45:47.327258 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcb7908-4ff7-4582-83f4-139d70993148 None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:45:47.335679 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:45:47.359605 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:45:47.419030 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff3a611-24c2-4b1a-b3ae-6f19c6b47ecf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:45:47.419595 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff3a611-24c2-4b1a-b3ae-6f19c6b47ecf None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:47.419929 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff3a611-24c2-4b1a-b3ae-6f19c6b47ecf None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:47.420200 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff3a611-24c2-4b1a-b3ae-6f19c6b47ecf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:47.439459 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff3a611-24c2-4b1a-b3ae-6f19c6b47ecf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 15:45:47.445275 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application [None req-1ff3a611-24c2-4b1a-b3ae-6f19c6b47ecf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 15:45:47.445275 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:45:47.445275 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:45:47.445275 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:45:47.445275 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:45:47.445275 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:45:47.445275 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:45:47.445275 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:45:47.445275 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:45:47.445275 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:45:47.445275 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:45:47.445275 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:45:47.445275 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 15:45:47.445275 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 15:45:47.445275 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 15:45:47.445275 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 15:45:47.446136 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 15:45:47.446136 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 15:45:47.446136 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 15:45:47.446136 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 15:45:47.446136 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:45:47.446136 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:45:47.446136 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:45:47.446136 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:45:47.446136 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:45:47.446136 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:45:47.446136 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:45:47.446136 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application with Lock( Aug 30 15:45:47.446136 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:45:47.446136 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:45:47.446136 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:45:47.446136 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:45:47.447150 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:45:47.447150 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:45:47.447150 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:45:47.447150 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:45:47.447150 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 15:45:47.447150 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 15:45:47.447150 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 15:45:47.447150 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 15:45:47.447150 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 15:45:47.447150 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 15:45:47.447150 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 15:45:47.447150 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Aug 30 15:45:47.447150 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 77/156] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Aug 30 15:45:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 15:45:47.448287 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbd1ede-f7f1-406a-b187-edff9df93b1f None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:47.448385 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbd1ede-f7f1-406a-b187-edff9df93b1f None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:47.448598 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbd1ede-f7f1-406a-b187-edff9df93b1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:47.476649 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd21d253-fa53-47f7-9b38-fbb1f92d1ac7 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:45:47.492316 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-2fbd1ede-f7f1-406a-b187-edff9df93b1f None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:45:47.494784 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-2fbd1ede-f7f1-406a-b187-edff9df93b1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:45:47.539057 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdcb430-a414-4e18-b294-775c7468b04d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:45:47.539685 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdcb430-a414-4e18-b294-775c7468b04d None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:47.539963 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdcb430-a414-4e18-b294-775c7468b04d None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:47.540228 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdcb430-a414-4e18-b294-775c7468b04d None admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:47.540915 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fdcb430-a414-4e18-b294-775c7468b04d None admin] RBAC: Adding query filter params (name=reader) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:45:47.541252 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fdcb430-a414-4e18-b294-775c7468b04d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:45:47.542922 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fdcb430-a414-4e18-b294-775c7468b04d None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:45:47.559213 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 80/157] 10.222.0.20 () {56 vars in 1101 bytes} [Tue Aug 30 15:45:47 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:45:47.560504 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215b2ed3-8a09-4bc5-b041-7bb52231aa23 None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:47.560789 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215b2ed3-8a09-4bc5-b041-7bb52231aa23 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:47.561061 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215b2ed3-8a09-4bc5-b041-7bb52231aa23 None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:47.575209 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 81/158] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 30 15:45:47 2022] GET /identity => generated 270 bytes in 15 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 15:45:47.575923 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215b2ed3-8a09-4bc5-b041-7bb52231aa23 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:45:47.627545 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 78/159] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 30 15:45:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 15:45:47.630586 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbd1ede-f7f1-406a-b187-edff9df93b1f None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:45:47.768989 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:45:47.777199 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215b2ed3-8a09-4bc5-b041-7bb52231aa23 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:45:47.807876 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:45:47.819078 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbd1ede-f7f1-406a-b187-edff9df93b1f None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:45:47.875669 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b8b0b2-eb2b-4093-9ed1-a191b22d7712 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:45:47.876345 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b8b0b2-eb2b-4093-9ed1-a191b22d7712 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:47.876630 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b8b0b2-eb2b-4093-9ed1-a191b22d7712 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:47.876906 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b8b0b2-eb2b-4093-9ed1-a191b22d7712 None admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:47.877616 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b8b0b2-eb2b-4093-9ed1-a191b22d7712 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:45:47.877919 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b8b0b2-eb2b-4093-9ed1-a191b22d7712 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:45:47.879328 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b8b0b2-eb2b-4093-9ed1-a191b22d7712 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:45:47.898584 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 82/160] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Aug 30 15:45:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:45:47.899818 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782b34f8-bd61-4a0b-b478-f92fa66db55f None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:47.900098 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782b34f8-bd61-4a0b-b478-f92fa66db55f None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:47.900365 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782b34f8-bd61-4a0b-b478-f92fa66db55f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:47.909938 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba924cf-df81-4d6a-89c8-6b1a09fbb12b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:45:47.910597 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba924cf-df81-4d6a-89c8-6b1a09fbb12b None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:47.910880 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba924cf-df81-4d6a-89c8-6b1a09fbb12b None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:47.911158 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba924cf-df81-4d6a-89c8-6b1a09fbb12b None admin] PATH_INFO: `/v3/users/4f46cc84bf04497e86d8762d42fd9a12` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:47.912341 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba924cf-df81-4d6a-89c8-6b1a09fbb12b None admin] RBAC: Authorizing `identity:get_user(user_id=4f46cc84bf04497e86d8762d42fd9a12)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:45:47.913860 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba924cf-df81-4d6a-89c8-6b1a09fbb12b None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:45:47.915457 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 79/161] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Aug 30 15:45:47 2022] GET /identity/v3/users/4f46cc84bf04497e86d8762d42fd9a12 => generated 289 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:45:47.920448 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba924cf-df81-4d6a-89c8-6b1a09fbb12b None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:45:47.924358 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-782b34f8-bd61-4a0b-b478-f92fa66db55f None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:45:47.926057 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-782b34f8-bd61-4a0b-b478-f92fa66db55f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:45:48.057364 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 83/162] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 30 15:45:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 15:45:48.058512 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782b34f8-bd61-4a0b-b478-f92fa66db55f None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:45:48.108846 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba924cf-df81-4d6a-89c8-6b1a09fbb12b None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:45:48.176811 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0da3792-c7fd-4b52-894d-6ddcff585fa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:45:48.177988 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0da3792-c7fd-4b52-894d-6ddcff585fa8 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:48.178470 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0da3792-c7fd-4b52-894d-6ddcff585fa8 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:48.178923 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0da3792-c7fd-4b52-894d-6ddcff585fa8 None admin] PATH_INFO: `/v3/users/8b9160d3cbae47a48dc9dee882da999b` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:48.180939 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0da3792-c7fd-4b52-894d-6ddcff585fa8 None admin] RBAC: Authorizing `identity:get_user(user_id=8b9160d3cbae47a48dc9dee882da999b)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:45:48.182617 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0da3792-c7fd-4b52-894d-6ddcff585fa8 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:45:48.185295 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 80/163] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Aug 30 15:45:47 2022] GET /identity/v3/users/8b9160d3cbae47a48dc9dee882da999b => generated 247 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:45:48.192896 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0da3792-c7fd-4b52-894d-6ddcff585fa8 None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:45:48.236691 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:45:48.238820 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:45:48.247938 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782b34f8-bd61-4a0b-b478-f92fa66db55f None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:45:48.293826 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ebc1db-973e-46b6-ab36-668f22b07e98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:45:48.294428 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ebc1db-973e-46b6-ab36-668f22b07e98 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:48.294984 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ebc1db-973e-46b6-ab36-668f22b07e98 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:48.295254 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ebc1db-973e-46b6-ab36-668f22b07e98 None admin] PATH_INFO: `/v3/projects/2a4e4930fe3247d3b04ad23013176d45` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:48.296865 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ebc1db-973e-46b6-ab36-668f22b07e98 None admin] RBAC: Authorizing `identity:get_project(project_id=2a4e4930fe3247d3b04ad23013176d45)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:45:48.298525 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ebc1db-973e-46b6-ab36-668f22b07e98 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:45:48.300548 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 84/164] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Aug 30 15:45:48 2022] GET /identity/v3/projects/2a4e4930fe3247d3b04ad23013176d45 => generated 298 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:45:48.320792 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0da3792-c7fd-4b52-894d-6ddcff585fa8 None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:45:48.326020 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ebc1db-973e-46b6-ab36-668f22b07e98 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:45:48.359030 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643dcbf6-fd5e-4b6a-b71b-fc553c072f4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:45:48.360109 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643dcbf6-fd5e-4b6a-b71b-fc553c072f4b None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:48.360649 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643dcbf6-fd5e-4b6a-b71b-fc553c072f4b None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:48.361112 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643dcbf6-fd5e-4b6a-b71b-fc553c072f4b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:48.437408 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643dcbf6-fd5e-4b6a-b71b-fc553c072f4b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:45:48.439241 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643dcbf6-fd5e-4b6a-b71b-fc553c072f4b None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:45:48.480653 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:45:48.545154 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ebc1db-973e-46b6-ab36-668f22b07e98 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:45:48.548244 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 81/165] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Aug 30 15:45:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 30 15:45:48.549470 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643dcbf6-fd5e-4b6a-b71b-fc553c072f4b None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:45:48.610582 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ed7cfc-d0e8-41ef-9b90-187ae6652890 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:45:48.611185 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ed7cfc-d0e8-41ef-9b90-187ae6652890 None admin] REQUEST_METHOD: `PUT` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:48.611462 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ed7cfc-d0e8-41ef-9b90-187ae6652890 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:48.611728 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ed7cfc-d0e8-41ef-9b90-187ae6652890 None admin] PATH_INFO: `/v3/projects/2a4e4930fe3247d3b04ad23013176d45/users/4f46cc84bf04497e86d8762d42fd9a12/roles/5d293e67c8e747e6910e95cf65e56633` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:48.613712 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ed7cfc-d0e8-41ef-9b90-187ae6652890 None admin] RBAC: Authorizing `identity:create_grant(project_id=2a4e4930fe3247d3b04ad23013176d45, user_id=4f46cc84bf04497e86d8762d42fd9a12, role_id=5d293e67c8e747e6910e95cf65e56633)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:45:48.615179 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ed7cfc-d0e8-41ef-9b90-187ae6652890 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:45:48.651366 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 85/166] 10.222.0.20 () {58 vars in 1439 bytes} [Tue Aug 30 15:45:48 2022] PUT /identity/v3/projects/2a4e4930fe3247d3b04ad23013176d45/users/4f46cc84bf04497e86d8762d42fd9a12/roles/5d293e67c8e747e6910e95cf65e56633 => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:45:48.652100 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ed7cfc-d0e8-41ef-9b90-187ae6652890 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:45:48.665156 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643dcbf6-fd5e-4b6a-b71b-fc553c072f4b None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:45:48.700074 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:45:48.705825 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889198ac-b5ac-4011-b835-3d2e1ac917e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:45:48.706799 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889198ac-b5ac-4011-b835-3d2e1ac917e4 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:48.707334 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889198ac-b5ac-4011-b835-3d2e1ac917e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:48.707764 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889198ac-b5ac-4011-b835-3d2e1ac917e4 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:48.716250 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:45:48.722657 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889198ac-b5ac-4011-b835-3d2e1ac917e4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 15:45:48.730973 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application [None req-889198ac-b5ac-4011-b835-3d2e1ac917e4 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 15:45:48.730973 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:45:48.730973 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:45:48.730973 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:45:48.730973 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:45:48.730973 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:45:48.730973 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:45:48.730973 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:45:48.730973 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:45:48.730973 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:45:48.730973 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:45:48.730973 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:45:48.730973 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 15:45:48.730973 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 15:45:48.730973 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 15:45:48.730973 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 15:45:48.732055 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 15:45:48.732055 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 15:45:48.732055 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 15:45:48.732055 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 15:45:48.732055 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:45:48.732055 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:45:48.732055 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:45:48.732055 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:45:48.732055 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:45:48.732055 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:45:48.732055 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:45:48.732055 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application with Lock( Aug 30 15:45:48.732055 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:45:48.732055 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:45:48.732055 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:45:48.732055 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:45:48.732996 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:45:48.732996 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:45:48.732996 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:45:48.732996 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:45:48.732996 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 15:45:48.732996 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 15:45:48.732996 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 15:45:48.732996 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 15:45:48.732996 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 15:45:48.732996 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 15:45:48.732996 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 15:45:48.732996 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Aug 30 15:45:48.735815 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 82/167] 10.222.0.20 () {56 vars in 1114 bytes} [Tue Aug 30 15:45:48 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 15:45:48.738374 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36da5a67-b611-47aa-bb7d-bcf3758dfd75 None None] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:48.739487 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36da5a67-b611-47aa-bb7d-bcf3758dfd75 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:48.739879 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36da5a67-b611-47aa-bb7d-bcf3758dfd75 None None] PATH_INFO: `/` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:48.741502 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 83/168] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 30 15:45:48 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 15:45:48.742471 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36da5a67-b611-47aa-bb7d-bcf3758dfd75 None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:45:48.855248 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36da5a67-b611-47aa-bb7d-bcf3758dfd75 None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:45:48.883937 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ed7cfc-d0e8-41ef-9b90-187ae6652890 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:45:48.893946 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62f1429-6497-4a4b-9889-2e818c22ddb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:45:48.894592 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62f1429-6497-4a4b-9889-2e818c22ddb5 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:48.894870 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62f1429-6497-4a4b-9889-2e818c22ddb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:48.895140 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62f1429-6497-4a4b-9889-2e818c22ddb5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:48.907224 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62f1429-6497-4a4b-9889-2e818c22ddb5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 15:45:48.912629 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application [None req-c62f1429-6497-4a4b-9889-2e818c22ddb5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 30 15:45:48.912629 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:45:48.912629 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:45:48.912629 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:45:48.912629 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:45:48.912629 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:45:48.912629 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:45:48.912629 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:45:48.912629 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:45:48.912629 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:45:48.912629 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:45:48.912629 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:45:48.912629 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 15:45:48.912629 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 15:45:48.912629 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 15:45:48.912629 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 15:45:48.913531 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 15:45:48.913531 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 15:45:48.913531 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 15:45:48.913531 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 15:45:48.913531 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:45:48.913531 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:45:48.913531 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:45:48.913531 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:45:48.913531 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:45:48.913531 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:45:48.913531 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:45:48.913531 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application with Lock( Aug 30 15:45:48.913531 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:45:48.913531 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:45:48.913531 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:45:48.913531 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:45:48.914377 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:45:48.914377 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:45:48.914377 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:45:48.914377 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:45:48.914377 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 15:45:48.914377 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 15:45:48.914377 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 15:45:48.914377 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 15:45:48.914377 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 15:45:48.914377 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 15:45:48.914377 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 30 15:45:48.914377 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Aug 30 15:45:48.914984 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 84/169] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Aug 30 15:45:48 2022] GET /identity/v3/roles/member => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 15:45:48.915702 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c050993-d01e-4a0d-83ff-e012bea5a15d None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:48.920031 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c050993-d01e-4a0d-83ff-e012bea5a15d None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:48.920031 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c050993-d01e-4a0d-83ff-e012bea5a15d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:48.930477 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fa69f2-606b-48be-8aae-096d60090109 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:45:48.931324 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fa69f2-606b-48be-8aae-096d60090109 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:48.931610 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fa69f2-606b-48be-8aae-096d60090109 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:48.931880 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fa69f2-606b-48be-8aae-096d60090109 None admin] PATH_INFO: `/v3/projects/2a4e4930fe3247d3b04ad23013176d45` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:48.933167 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3fa69f2-606b-48be-8aae-096d60090109 None admin] RBAC: Authorizing `identity:get_project(project_id=2a4e4930fe3247d3b04ad23013176d45)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:45:48.934676 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3fa69f2-606b-48be-8aae-096d60090109 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:45:48.936086 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-4c050993-d01e-4a0d-83ff-e012bea5a15d None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:45:48.936322 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 86/170] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Aug 30 15:45:48 2022] GET /identity/v3/projects/2a4e4930fe3247d3b04ad23013176d45 => generated 298 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:45:48.937159 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fa69f2-606b-48be-8aae-096d60090109 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:45:48.938175 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-4c050993-d01e-4a0d-83ff-e012bea5a15d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:45:48.993698 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:45:49.021388 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 85/171] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 30 15:45:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 15:45:49.023223 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c050993-d01e-4a0d-83ff-e012bea5a15d None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:45:49.136339 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c050993-d01e-4a0d-83ff-e012bea5a15d None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:45:49.153323 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:45:49.172728 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49aa640b-fe75-4bb5-9b1a-22cc04c0524e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:45:49.173382 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49aa640b-fe75-4bb5-9b1a-22cc04c0524e None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:49.173663 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49aa640b-fe75-4bb5-9b1a-22cc04c0524e None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:49.173932 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49aa640b-fe75-4bb5-9b1a-22cc04c0524e None admin] PATH_INFO: `/v3/roles` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:49.174832 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49aa640b-fe75-4bb5-9b1a-22cc04c0524e None admin] RBAC: Adding query filter params (name=member) {{(pid=81512) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:45:49.175141 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49aa640b-fe75-4bb5-9b1a-22cc04c0524e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:45:49.176547 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49aa640b-fe75-4bb5-9b1a-22cc04c0524e None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:45:49.185893 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 86/172] 10.222.0.20 () {56 vars in 1101 bytes} [Tue Aug 30 15:45:49 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:45:49.186607 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49aa640b-fe75-4bb5-9b1a-22cc04c0524e None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:45:49.193192 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fa69f2-606b-48be-8aae-096d60090109 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:45:49.233601 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:45:49.273670 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c5d31b-bc76-4d9b-9db2-ea1e7f72e8ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:45:49.275003 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c5d31b-bc76-4d9b-9db2-ea1e7f72e8ed None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:49.275555 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c5d31b-bc76-4d9b-9db2-ea1e7f72e8ed None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:49.276148 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c5d31b-bc76-4d9b-9db2-ea1e7f72e8ed None admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:49.277015 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c5d31b-bc76-4d9b-9db2-ea1e7f72e8ed None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:45:49.277551 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c5d31b-bc76-4d9b-9db2-ea1e7f72e8ed None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:45:49.279254 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c5d31b-bc76-4d9b-9db2-ea1e7f72e8ed None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:45:49.288501 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 87/173] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Aug 30 15:45:48 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 352 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:45:49.289981 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c5d31b-bc76-4d9b-9db2-ea1e7f72e8ed None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:45:49.292743 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49aa640b-fe75-4bb5-9b1a-22cc04c0524e None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:45:49.337738 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8a1301-f863-4b89-ba03-2e0ed6882127 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:45:49.338361 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8a1301-f863-4b89-ba03-2e0ed6882127 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:49.338642 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8a1301-f863-4b89-ba03-2e0ed6882127 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:49.338913 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8a1301-f863-4b89-ba03-2e0ed6882127 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:49.339819 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8a1301-f863-4b89-ba03-2e0ed6882127 None admin] RBAC: Adding query filter params (role.id=f265386950a641ac8413b53eda4ae34d, scope.project.id=2a4e4930fe3247d3b04ad23013176d45, user.id=8b9160d3cbae47a48dc9dee882da999b) {{(pid=81512) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:45:49.340131 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8a1301-f863-4b89-ba03-2e0ed6882127 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:45:49.342286 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8a1301-f863-4b89-ba03-2e0ed6882127 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:45:49.349647 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 87/174] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Aug 30 15:45:49 2022] GET /identity/v3/role_assignments?user.id=8b9160d3cbae47a48dc9dee882da999b&scope.project.id=2a4e4930fe3247d3b04ad23013176d45&role.id=f265386950a641ac8413b53eda4ae34d => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:45:49.353195 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8a1301-f863-4b89-ba03-2e0ed6882127 None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:45:49.433599 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c5d31b-bc76-4d9b-9db2-ea1e7f72e8ed None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:45:49.471849 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8a1301-f863-4b89-ba03-2e0ed6882127 None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:45:49.481554 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:45:49.505430 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5bb4d8-5b3a-47c3-9e16-d3d4631012ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:45:49.513615 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ce1106-831c-40d4-a2c9-20525d3f56a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:45:49.514015 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5bb4d8-5b3a-47c3-9e16-d3d4631012ac None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:49.514535 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5bb4d8-5b3a-47c3-9e16-d3d4631012ac None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:49.514786 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ce1106-831c-40d4-a2c9-20525d3f56a0 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:49.515093 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5bb4d8-5b3a-47c3-9e16-d3d4631012ac None admin] PATH_INFO: `/v3/users/8249204af55844a180f2d1d09c8b1f59` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:49.515230 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ce1106-831c-40d4-a2c9-20525d3f56a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:49.515685 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ce1106-831c-40d4-a2c9-20525d3f56a0 None admin] PATH_INFO: `/v3/users/8249204af55844a180f2d1d09c8b1f59` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:49.516908 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5bb4d8-5b3a-47c3-9e16-d3d4631012ac None admin] RBAC: Authorizing `identity:get_user(user_id=8249204af55844a180f2d1d09c8b1f59)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:45:49.517021 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ce1106-831c-40d4-a2c9-20525d3f56a0 None admin] RBAC: Authorizing `identity:get_user(user_id=8249204af55844a180f2d1d09c8b1f59)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:45:49.518619 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ce1106-831c-40d4-a2c9-20525d3f56a0 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:45:49.518853 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5bb4d8-5b3a-47c3-9e16-d3d4631012ac None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:45:49.520602 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 88/175] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Aug 30 15:45:49 2022] GET /identity/v3/users/8249204af55844a180f2d1d09c8b1f59 => generated 275 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:45:49.521084 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 88/176] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Aug 30 15:45:49 2022] GET /identity/v3/users/8249204af55844a180f2d1d09c8b1f59 => generated 275 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:45:49.532465 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5bb4d8-5b3a-47c3-9e16-d3d4631012ac None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:45:49.532678 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ce1106-831c-40d4-a2c9-20525d3f56a0 None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:45:49.563282 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:45:49.666853 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ce1106-831c-40d4-a2c9-20525d3f56a0 None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:45:49.706353 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fec81f-42f1-41f9-bc45-2fa90798550e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:45:49.706912 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fec81f-42f1-41f9-bc45-2fa90798550e None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:49.707935 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fec81f-42f1-41f9-bc45-2fa90798550e None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:49.708208 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fec81f-42f1-41f9-bc45-2fa90798550e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:49.743418 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5bb4d8-5b3a-47c3-9e16-d3d4631012ac None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:45:49.769016 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:45:49.786348 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aebb3a8-48e2-45bc-97ae-9a6e6683e411 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:45:49.787385 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aebb3a8-48e2-45bc-97ae-9a6e6683e411 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:49.787671 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aebb3a8-48e2-45bc-97ae-9a6e6683e411 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:49.787940 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aebb3a8-48e2-45bc-97ae-9a6e6683e411 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:49.834271 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fec81f-42f1-41f9-bc45-2fa90798550e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:45:49.835781 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fec81f-42f1-41f9-bc45-2fa90798550e None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:45:49.868276 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aebb3a8-48e2-45bc-97ae-9a6e6683e411 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:45:49.870074 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aebb3a8-48e2-45bc-97ae-9a6e6683e411 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:45:49.890325 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:45:49.996350 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 89/177] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Aug 30 15:45:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 460 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 15:45:49.996350 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aebb3a8-48e2-45bc-97ae-9a6e6683e411 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:45:49.998160 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 89/178] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Aug 30 15:45:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 466 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 30 15:45:49.999469 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7813aa-87e3-4b04-a547-2da91b6bb903 None None] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:49.999784 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7813aa-87e3-4b04-a547-2da91b6bb903 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:50.000055 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7813aa-87e3-4b04-a547-2da91b6bb903 None None] PATH_INFO: `/` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:50.002775 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 90/179] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 30 15:45:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 15:45:50.002775 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25146c61-c8c1-4664-b76e-c14ae81ab60d None None] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:50.002775 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25146c61-c8c1-4664-b76e-c14ae81ab60d None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:50.003023 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25146c61-c8c1-4664-b76e-c14ae81ab60d None None] PATH_INFO: `/` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:50.004889 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 91/180] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 30 15:45:50 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 15:45:50.004889 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25146c61-c8c1-4664-b76e-c14ae81ab60d None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:45:50.088728 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:45:50.109890 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aebb3a8-48e2-45bc-97ae-9a6e6683e411 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:45:50.116853 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25146c61-c8c1-4664-b76e-c14ae81ab60d None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:45:50.128301 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:45:50.149077 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115f4371-048d-40bf-b9d5-8109fd36bb9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:45:50.149747 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115f4371-048d-40bf-b9d5-8109fd36bb9a None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:50.150021 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115f4371-048d-40bf-b9d5-8109fd36bb9a None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:50.150288 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115f4371-048d-40bf-b9d5-8109fd36bb9a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:50.157956 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2364623c-ce2e-40e8-9168-1d3b8cb40ee7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:45:50.159199 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2364623c-ce2e-40e8-9168-1d3b8cb40ee7 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:50.159622 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2364623c-ce2e-40e8-9168-1d3b8cb40ee7 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:50.160038 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2364623c-ce2e-40e8-9168-1d3b8cb40ee7 None admin] PATH_INFO: `/v3/projects/8ed1a9892a9a428a92a900c9ce92f2b0` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:50.161777 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115f4371-048d-40bf-b9d5-8109fd36bb9a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 15:45:50.161974 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2364623c-ce2e-40e8-9168-1d3b8cb40ee7 None admin] RBAC: Authorizing `identity:get_project(project_id=8ed1a9892a9a428a92a900c9ce92f2b0)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:45:50.163607 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2364623c-ce2e-40e8-9168-1d3b8cb40ee7 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:45:50.165934 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 92/181] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Aug 30 15:45:50 2022] GET /identity/v3/projects/8ed1a9892a9a428a92a900c9ce92f2b0 => generated 312 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:45:50.166924 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-115f4371-048d-40bf-b9d5-8109fd36bb9a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 15:45:50.166924 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:45:50.166924 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:45:50.166924 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:45:50.166924 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:45:50.166924 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:45:50.166924 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:45:50.166924 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:45:50.166924 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:45:50.166924 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:45:50.166924 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:45:50.166924 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:45:50.166924 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 15:45:50.166924 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 15:45:50.166924 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 15:45:50.166924 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 15:45:50.167822 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 15:45:50.167822 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 15:45:50.167822 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 15:45:50.167822 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 15:45:50.167822 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:45:50.167822 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:45:50.167822 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:45:50.167822 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:45:50.167822 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:45:50.167822 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:45:50.167822 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:45:50.167822 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Aug 30 15:45:50.167822 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:45:50.167822 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:45:50.167822 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:45:50.167822 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:45:50.168630 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:45:50.168630 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:45:50.168630 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:45:50.168630 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:45:50.168630 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 15:45:50.168630 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 15:45:50.168630 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 15:45:50.168630 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 15:45:50.168630 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 15:45:50.168630 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 15:45:50.168630 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 15:45:50.168630 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Aug 30 15:45:50.169274 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2364623c-ce2e-40e8-9168-1d3b8cb40ee7 None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:45:50.169346 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 90/182] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Aug 30 15:45:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 15:45:50.169782 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fda6981-9dde-4717-a1a8-c64e2fd29507 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:50.170060 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fda6981-9dde-4717-a1a8-c64e2fd29507 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:50.170619 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fda6981-9dde-4717-a1a8-c64e2fd29507 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:50.192318 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-1fda6981-9dde-4717-a1a8-c64e2fd29507 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:45:50.193926 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-1fda6981-9dde-4717-a1a8-c64e2fd29507 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:45:50.264223 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 91/183] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 30 15:45:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 15:45:50.265521 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfed9b8-9d4c-4c7e-83dc-750b66a21969 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:50.265809 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfed9b8-9d4c-4c7e-83dc-750b66a21969 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:50.266050 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfed9b8-9d4c-4c7e-83dc-750b66a21969 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:50.283117 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2364623c-ce2e-40e8-9168-1d3b8cb40ee7 None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:45:50.287306 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-cbfed9b8-9d4c-4c7e-83dc-750b66a21969 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:45:50.289058 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-cbfed9b8-9d4c-4c7e-83dc-750b66a21969 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:45:50.319627 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fee142-0a32-411b-aa54-83f2c96f4c6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:45:50.320194 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fee142-0a32-411b-aa54-83f2c96f4c6c None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:50.320472 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fee142-0a32-411b-aa54-83f2c96f4c6c None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:50.320794 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fee142-0a32-411b-aa54-83f2c96f4c6c None admin] PATH_INFO: `/v3/projects/2a4e4930fe3247d3b04ad23013176d45` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:50.321999 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7fee142-0a32-411b-aa54-83f2c96f4c6c None admin] RBAC: Authorizing `identity:get_project(project_id=2a4e4930fe3247d3b04ad23013176d45)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:45:50.323844 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7fee142-0a32-411b-aa54-83f2c96f4c6c None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:45:50.330012 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 93/184] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Aug 30 15:45:50 2022] GET /identity/v3/projects/2a4e4930fe3247d3b04ad23013176d45 => generated 298 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:45:50.330012 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fee142-0a32-411b-aa54-83f2c96f4c6c None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:45:50.342261 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:45:50.379607 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 92/185] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 30 15:45:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 15:45:50.381506 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfed9b8-9d4c-4c7e-83dc-750b66a21969 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:45:50.425375 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:45:50.438243 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fee142-0a32-411b-aa54-83f2c96f4c6c None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:45:50.487402 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2c147b-b826-4717-91ad-df5b781991b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:45:50.488018 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2c147b-b826-4717-91ad-df5b781991b3 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:50.488300 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2c147b-b826-4717-91ad-df5b781991b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:50.488566 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2c147b-b826-4717-91ad-df5b781991b3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:50.489817 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2c147b-b826-4717-91ad-df5b781991b3 None admin] RBAC: Adding query filter params (role.id=df8f9f844b1b439181d75a07c1a93418, scope.project.id=8ed1a9892a9a428a92a900c9ce92f2b0, user.id=8249204af55844a180f2d1d09c8b1f59) {{(pid=81512) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:45:50.489920 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2c147b-b826-4717-91ad-df5b781991b3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:45:50.492237 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2c147b-b826-4717-91ad-df5b781991b3 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:45:50.506399 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 94/186] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Aug 30 15:45:50 2022] GET /identity/v3/role_assignments?user.id=8249204af55844a180f2d1d09c8b1f59&scope.project.id=8ed1a9892a9a428a92a900c9ce92f2b0&role.id=df8f9f844b1b439181d75a07c1a93418 => generated 608 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:45:50.507225 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2c147b-b826-4717-91ad-df5b781991b3 None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:45:50.535036 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfed9b8-9d4c-4c7e-83dc-750b66a21969 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:45:50.574620 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11d0d5e-e09d-4f31-ab0a-6e3e47854275 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:45:50.575456 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11d0d5e-e09d-4f31-ab0a-6e3e47854275 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:50.575749 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11d0d5e-e09d-4f31-ab0a-6e3e47854275 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:50.576015 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11d0d5e-e09d-4f31-ab0a-6e3e47854275 None admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:50.576705 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11d0d5e-e09d-4f31-ab0a-6e3e47854275 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:45:50.577003 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11d0d5e-e09d-4f31-ab0a-6e3e47854275 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:45:50.578518 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11d0d5e-e09d-4f31-ab0a-6e3e47854275 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:45:50.599623 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 93/187] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Aug 30 15:45:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:45:50.611627 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11d0d5e-e09d-4f31-ab0a-6e3e47854275 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:45:50.668807 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:45:50.726925 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2c147b-b826-4717-91ad-df5b781991b3 None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:45:50.768136 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcff8d9-51a8-420f-92de-45fca8c3c46c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:45:50.768766 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcff8d9-51a8-420f-92de-45fca8c3c46c None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:50.769048 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcff8d9-51a8-420f-92de-45fca8c3c46c None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:50.769355 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcff8d9-51a8-420f-92de-45fca8c3c46c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:50.770092 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcff8d9-51a8-420f-92de-45fca8c3c46c None admin] RBAC: Adding query filter params (role.id=6b4385c3b449456d848c9abb16766d25, scope.project.id=2a4e4930fe3247d3b04ad23013176d45, user.id=8249204af55844a180f2d1d09c8b1f59) {{(pid=81512) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:45:50.770401 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcff8d9-51a8-420f-92de-45fca8c3c46c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:45:50.771103 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:45:50.771870 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcff8d9-51a8-420f-92de-45fca8c3c46c None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:45:50.776500 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11d0d5e-e09d-4f31-ab0a-6e3e47854275 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:45:50.778618 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 95/188] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Aug 30 15:45:50 2022] GET /identity/v3/role_assignments?user.id=8249204af55844a180f2d1d09c8b1f59&scope.project.id=2a4e4930fe3247d3b04ad23013176d45&role.id=6b4385c3b449456d848c9abb16766d25 => generated 608 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:45:50.779473 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcff8d9-51a8-420f-92de-45fca8c3c46c None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:45:50.823655 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fc2c74-ab9a-419d-82cb-3bdbdc696418 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:45:50.824276 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fc2c74-ab9a-419d-82cb-3bdbdc696418 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:50.824555 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fc2c74-ab9a-419d-82cb-3bdbdc696418 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:50.824824 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fc2c74-ab9a-419d-82cb-3bdbdc696418 None admin] PATH_INFO: `/v3/users/bc54674c67dc43898519b1e2d04e9aa3` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:50.840888 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48fc2c74-ab9a-419d-82cb-3bdbdc696418 None admin] RBAC: Authorizing `identity:get_user(user_id=bc54674c67dc43898519b1e2d04e9aa3)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:45:50.842577 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48fc2c74-ab9a-419d-82cb-3bdbdc696418 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:45:50.848096 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 94/189] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Aug 30 15:45:50 2022] GET /identity/v3/users/bc54674c67dc43898519b1e2d04e9aa3 => generated 283 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 15:45:50.849288 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fc2c74-ab9a-419d-82cb-3bdbdc696418 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:45:50.914466 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcff8d9-51a8-420f-92de-45fca8c3c46c None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:45:50.962942 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f14ef8-25af-4cff-9296-39820d5e4dff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:45:50.963928 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f14ef8-25af-4cff-9296-39820d5e4dff None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:50.964425 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f14ef8-25af-4cff-9296-39820d5e4dff None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:50.964970 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f14ef8-25af-4cff-9296-39820d5e4dff None admin] PATH_INFO: `/v3/domains/default` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:50.966303 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f14ef8-25af-4cff-9296-39820d5e4dff None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:45:50.966606 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fc2c74-ab9a-419d-82cb-3bdbdc696418 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:45:50.969320 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f14ef8-25af-4cff-9296-39820d5e4dff None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:45:50.971017 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 96/190] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 30 15:45:50 2022] GET /identity/v3/domains/default => generated 199 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:45:50.974135 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f14ef8-25af-4cff-9296-39820d5e4dff None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:45:50.987115 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:45:51.007135 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12dcdd22-bd47-4cd2-9b8c-3cc36770d04c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:45:51.007879 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dcdd22-bd47-4cd2-9b8c-3cc36770d04c None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:51.008160 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dcdd22-bd47-4cd2-9b8c-3cc36770d04c None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:51.008432 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dcdd22-bd47-4cd2-9b8c-3cc36770d04c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:51.020522 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12dcdd22-bd47-4cd2-9b8c-3cc36770d04c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 15:45:51.025511 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-12dcdd22-bd47-4cd2-9b8c-3cc36770d04c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 15:45:51.025511 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:45:51.025511 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:45:51.025511 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:45:51.025511 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:45:51.025511 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:45:51.025511 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:45:51.025511 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:45:51.025511 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:45:51.025511 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:45:51.025511 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:45:51.025511 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:45:51.025511 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 15:45:51.025511 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 15:45:51.025511 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 15:45:51.025511 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 15:45:51.026575 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 15:45:51.026575 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 15:45:51.026575 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 15:45:51.026575 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 15:45:51.026575 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:45:51.026575 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:45:51.026575 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:45:51.026575 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:45:51.026575 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:45:51.026575 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:45:51.026575 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:45:51.026575 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Aug 30 15:45:51.026575 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:45:51.026575 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:45:51.026575 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:45:51.026575 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:45:51.027754 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:45:51.027754 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:45:51.027754 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:45:51.027754 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:45:51.027754 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 15:45:51.027754 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 15:45:51.027754 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 15:45:51.027754 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 15:45:51.027754 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 15:45:51.027754 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 15:45:51.027754 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 15:45:51.027754 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Aug 30 15:45:51.027754 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 95/191] 10.222.0.20 () {56 vars in 1114 bytes} [Tue Aug 30 15:45:50 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 15:45:51.027754 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12dcdd22-bd47-4cd2-9b8c-3cc36770d04c None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:45:51.056928 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:45:51.099457 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f14ef8-25af-4cff-9296-39820d5e4dff None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:45:51.139500 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf198b2b-ff87-4c3d-85bd-2e5dc5cff784 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:45:51.140119 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf198b2b-ff87-4c3d-85bd-2e5dc5cff784 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:51.140427 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf198b2b-ff87-4c3d-85bd-2e5dc5cff784 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:51.140698 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf198b2b-ff87-4c3d-85bd-2e5dc5cff784 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:51.157020 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12dcdd22-bd47-4cd2-9b8c-3cc36770d04c None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:45:51.190258 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5346d03-a072-4158-829d-586e2ac3a78c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:45:51.190867 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5346d03-a072-4158-829d-586e2ac3a78c None admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:51.191138 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5346d03-a072-4158-829d-586e2ac3a78c None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:51.191399 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5346d03-a072-4158-829d-586e2ac3a78c None admin] PATH_INFO: `/v3/users` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:51.192051 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5346d03-a072-4158-829d-586e2ac3a78c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:45:51.193731 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5346d03-a072-4158-829d-586e2ac3a78c None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:45:51.210194 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf198b2b-ff87-4c3d-85bd-2e5dc5cff784 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:45:51.211899 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf198b2b-ff87-4c3d-85bd-2e5dc5cff784 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:45:51.215943 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 96/192] 10.222.0.20 () {60 vars in 1132 bytes} [Tue Aug 30 15:45:51 2022] POST /identity/v3/users => generated 297 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 15:45:51.216766 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5346d03-a072-4158-829d-586e2ac3a78c None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:45:51.221512 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:45:51.297088 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:45:51.332894 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 97/193] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Aug 30 15:45:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 15:45:51.338093 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5346d03-a072-4158-829d-586e2ac3a78c None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:45:51.338238 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd26064-a385-4ac7-bca7-856719e855a0 None None] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:51.338238 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd26064-a385-4ac7-bca7-856719e855a0 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:51.338238 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd26064-a385-4ac7-bca7-856719e855a0 None None] PATH_INFO: `/` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:51.338238 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 98/194] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 30 15:45:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 15:45:51.342804 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd26064-a385-4ac7-bca7-856719e855a0 None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:45:51.375259 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0345a9-5500-4e2e-a9e8-3c0eb116c037 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:45:51.375798 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0345a9-5500-4e2e-a9e8-3c0eb116c037 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:51.376077 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0345a9-5500-4e2e-a9e8-3c0eb116c037 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:51.376347 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0345a9-5500-4e2e-a9e8-3c0eb116c037 None admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:51.377027 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0345a9-5500-4e2e-a9e8-3c0eb116c037 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:45:51.377364 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0345a9-5500-4e2e-a9e8-3c0eb116c037 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:45:51.378768 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0345a9-5500-4e2e-a9e8-3c0eb116c037 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:45:51.388089 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 97/195] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Aug 30 15:45:51 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:45:51.389322 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798c29a8-6ba8-402c-9347-cb7ebd95bd20 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:51.389606 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798c29a8-6ba8-402c-9347-cb7ebd95bd20 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:51.389877 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798c29a8-6ba8-402c-9347-cb7ebd95bd20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:51.408054 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-798c29a8-6ba8-402c-9347-cb7ebd95bd20 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:45:51.411071 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-798c29a8-6ba8-402c-9347-cb7ebd95bd20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:45:51.462310 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd26064-a385-4ac7-bca7-856719e855a0 None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:45:51.472498 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:45:51.488670 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 98/196] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 30 15:45:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 15:45:51.489653 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798c29a8-6ba8-402c-9347-cb7ebd95bd20 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:45:51.522202 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8747c82b-c2c8-4f39-90d3-b5419f358001 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:45:51.523810 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8747c82b-c2c8-4f39-90d3-b5419f358001 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:51.524612 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8747c82b-c2c8-4f39-90d3-b5419f358001 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:51.525318 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8747c82b-c2c8-4f39-90d3-b5419f358001 None admin] PATH_INFO: `/v3/projects/c3ff9ff008b04f21a0d68698cf06e6dc` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:51.527513 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8747c82b-c2c8-4f39-90d3-b5419f358001 None admin] RBAC: Authorizing `identity:get_project(project_id=c3ff9ff008b04f21a0d68698cf06e6dc)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:45:51.530439 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8747c82b-c2c8-4f39-90d3-b5419f358001 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:45:51.535454 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 99/197] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Aug 30 15:45:51 2022] GET /identity/v3/projects/c3ff9ff008b04f21a0d68698cf06e6dc => generated 302 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:45:51.542429 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8747c82b-c2c8-4f39-90d3-b5419f358001 None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:45:51.600758 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:45:51.658968 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798c29a8-6ba8-402c-9347-cb7ebd95bd20 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:45:51.698911 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3977a49e-775f-47e4-98d1-dbfc89548f7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:45:51.699705 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3977a49e-775f-47e4-98d1-dbfc89548f7d None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:51.700080 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3977a49e-775f-47e4-98d1-dbfc89548f7d None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:51.700352 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3977a49e-775f-47e4-98d1-dbfc89548f7d None admin] PATH_INFO: `/v3/users/bc54674c67dc43898519b1e2d04e9aa3` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:51.701539 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3977a49e-775f-47e4-98d1-dbfc89548f7d None admin] RBAC: Authorizing `identity:get_user(user_id=bc54674c67dc43898519b1e2d04e9aa3)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:45:51.703115 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3977a49e-775f-47e4-98d1-dbfc89548f7d None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:45:51.704819 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 99/198] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Aug 30 15:45:51 2022] GET /identity/v3/users/bc54674c67dc43898519b1e2d04e9aa3 => generated 283 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:45:51.710790 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3977a49e-775f-47e4-98d1-dbfc89548f7d None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:45:51.715763 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8747c82b-c2c8-4f39-90d3-b5419f358001 None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:45:51.764576 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b2b336-0e44-4041-8bb2-49a775c86fba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:45:51.765332 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b2b336-0e44-4041-8bb2-49a775c86fba None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:51.766882 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b2b336-0e44-4041-8bb2-49a775c86fba None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:51.767175 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b2b336-0e44-4041-8bb2-49a775c86fba None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:51.767925 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b2b336-0e44-4041-8bb2-49a775c86fba None admin] RBAC: Adding query filter params (role.id=f265386950a641ac8413b53eda4ae34d, scope.project.id=c3ff9ff008b04f21a0d68698cf06e6dc, user.id=bc54674c67dc43898519b1e2d04e9aa3) {{(pid=81512) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:45:51.768233 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b2b336-0e44-4041-8bb2-49a775c86fba None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:45:51.769708 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b2b336-0e44-4041-8bb2-49a775c86fba None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:45:51.785982 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 100/199] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Aug 30 15:45:51 2022] GET /identity/v3/role_assignments?user.id=bc54674c67dc43898519b1e2d04e9aa3&scope.project.id=c3ff9ff008b04f21a0d68698cf06e6dc&role.id=f265386950a641ac8413b53eda4ae34d => generated 259 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 15:45:51.807939 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b2b336-0e44-4041-8bb2-49a775c86fba None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:45:51.850996 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3977a49e-775f-47e4-98d1-dbfc89548f7d None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:45:51.875849 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:45:51.901713 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:45:51.911511 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0d8ae2-8157-4c3a-8b63-0b455c823c0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:45:51.912671 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0d8ae2-8157-4c3a-8b63-0b455c823c0f None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:51.913620 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0d8ae2-8157-4c3a-8b63-0b455c823c0f None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:51.915290 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0d8ae2-8157-4c3a-8b63-0b455c823c0f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:52.004669 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b2b336-0e44-4041-8bb2-49a775c86fba None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:45:52.044642 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0d8ae2-8157-4c3a-8b63-0b455c823c0f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:45:52.053449 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0d8ae2-8157-4c3a-8b63-0b455c823c0f None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:45:52.071819 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329ab140-60a3-47c8-9081-af9cbb56214e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:45:52.072796 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329ab140-60a3-47c8-9081-af9cbb56214e None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:52.073287 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329ab140-60a3-47c8-9081-af9cbb56214e None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:52.073835 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329ab140-60a3-47c8-9081-af9cbb56214e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:52.105727 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329ab140-60a3-47c8-9081-af9cbb56214e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 15:45:52.116183 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application [None req-329ab140-60a3-47c8-9081-af9cbb56214e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 30 15:45:52.116183 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:45:52.116183 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:45:52.116183 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:45:52.116183 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:45:52.116183 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:45:52.116183 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:45:52.116183 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:45:52.116183 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:45:52.116183 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:45:52.116183 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:45:52.116183 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:45:52.116183 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 15:45:52.116183 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 15:45:52.116183 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 15:45:52.116183 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 15:45:52.117090 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 15:45:52.117090 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 15:45:52.117090 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 15:45:52.117090 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 15:45:52.117090 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:45:52.117090 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:45:52.117090 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:45:52.117090 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:45:52.117090 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:45:52.117090 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:45:52.117090 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:45:52.117090 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application with Lock( Aug 30 15:45:52.117090 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:45:52.117090 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:45:52.117090 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:45:52.117090 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:45:52.117914 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:45:52.117914 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:45:52.117914 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:45:52.117914 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:45:52.117914 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 15:45:52.117914 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 15:45:52.117914 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 15:45:52.117914 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 15:45:52.117914 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 15:45:52.117914 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 15:45:52.117914 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 30 15:45:52.117914 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Aug 30 15:45:52.121506 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 101/200] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Aug 30 15:45:51 2022] GET /identity/v3/roles/reader => generated 84 bytes in 314 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 15:45:52.132974 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329ab140-60a3-47c8-9081-af9cbb56214e None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:45:52.275840 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 100/201] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Aug 30 15:45:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 565 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 30 15:45:52.279638 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0d8ae2-8157-4c3a-8b63-0b455c823c0f None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:45:52.381604 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:45:52.422233 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:45:52.433145 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329ab140-60a3-47c8-9081-af9cbb56214e None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:45:52.516466 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0d8ae2-8157-4c3a-8b63-0b455c823c0f None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:45:52.522522 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4cfd01-8f26-4e69-afc2-c488f976d147 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:45:52.523160 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4cfd01-8f26-4e69-afc2-c488f976d147 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:52.523438 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4cfd01-8f26-4e69-afc2-c488f976d147 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:52.523708 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4cfd01-8f26-4e69-afc2-c488f976d147 None admin] PATH_INFO: `/v3/roles` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:52.524396 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4cfd01-8f26-4e69-afc2-c488f976d147 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81512) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:45:52.524699 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4cfd01-8f26-4e69-afc2-c488f976d147 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:45:52.526186 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4cfd01-8f26-4e69-afc2-c488f976d147 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:45:52.552946 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 102/202] 10.222.0.20 () {56 vars in 1101 bytes} [Tue Aug 30 15:45:52 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 420 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:45:52.564839 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4cfd01-8f26-4e69-afc2-c488f976d147 None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:45:52.616897 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf10ccf-46a4-43fe-b6ef-44c9d4f7aec1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:45:52.619803 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf10ccf-46a4-43fe-b6ef-44c9d4f7aec1 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:52.622144 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf10ccf-46a4-43fe-b6ef-44c9d4f7aec1 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:52.622646 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf10ccf-46a4-43fe-b6ef-44c9d4f7aec1 None admin] PATH_INFO: `/v3/projects/c3ff9ff008b04f21a0d68698cf06e6dc` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:52.624582 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf10ccf-46a4-43fe-b6ef-44c9d4f7aec1 None admin] RBAC: Authorizing `identity:get_project(project_id=c3ff9ff008b04f21a0d68698cf06e6dc)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:45:52.633483 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf10ccf-46a4-43fe-b6ef-44c9d4f7aec1 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:45:52.638300 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 101/203] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Aug 30 15:45:52 2022] GET /identity/v3/projects/c3ff9ff008b04f21a0d68698cf06e6dc => generated 302 bytes in 359 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:45:52.645253 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf10ccf-46a4-43fe-b6ef-44c9d4f7aec1 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:45:52.744800 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4cfd01-8f26-4e69-afc2-c488f976d147 None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:45:52.793721 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:45:52.797749 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf10ccf-46a4-43fe-b6ef-44c9d4f7aec1 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:45:52.808624 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c282fe-3ffd-427e-be04-efc4ccb8bd66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:45:52.809296 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c282fe-3ffd-427e-be04-efc4ccb8bd66 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:52.809577 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c282fe-3ffd-427e-be04-efc4ccb8bd66 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:52.809848 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c282fe-3ffd-427e-be04-efc4ccb8bd66 None admin] PATH_INFO: `/v3/users/4f46cc84bf04497e86d8762d42fd9a12` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:52.811087 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c282fe-3ffd-427e-be04-efc4ccb8bd66 None admin] RBAC: Authorizing `identity:get_user(user_id=4f46cc84bf04497e86d8762d42fd9a12)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:45:52.812512 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c282fe-3ffd-427e-be04-efc4ccb8bd66 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:45:52.814324 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 103/204] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Aug 30 15:45:52 2022] GET /identity/v3/users/4f46cc84bf04497e86d8762d42fd9a12 => generated 289 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:45:52.821884 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c282fe-3ffd-427e-be04-efc4ccb8bd66 None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:45:52.844374 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bebf52a-597b-4183-b6c3-63a851d6c018 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:45:52.845052 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bebf52a-597b-4183-b6c3-63a851d6c018 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:52.845363 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bebf52a-597b-4183-b6c3-63a851d6c018 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:52.845636 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bebf52a-597b-4183-b6c3-63a851d6c018 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:52.846383 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bebf52a-597b-4183-b6c3-63a851d6c018 None admin] RBAC: Adding query filter params (role.id=6b4385c3b449456d848c9abb16766d25, scope.project.id=c3ff9ff008b04f21a0d68698cf06e6dc, user.id=bc54674c67dc43898519b1e2d04e9aa3) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:45:52.846693 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bebf52a-597b-4183-b6c3-63a851d6c018 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:45:52.848382 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bebf52a-597b-4183-b6c3-63a851d6c018 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:45:52.854443 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 102/205] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Aug 30 15:45:52 2022] GET /identity/v3/role_assignments?user.id=bc54674c67dc43898519b1e2d04e9aa3&scope.project.id=c3ff9ff008b04f21a0d68698cf06e6dc&role.id=6b4385c3b449456d848c9abb16766d25 => generated 259 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:45:52.876636 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:45:52.991045 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c282fe-3ffd-427e-be04-efc4ccb8bd66 None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:45:53.036906 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117583b7-4c4a-47f7-988e-dfe8cd9d3a12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:45:53.037560 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117583b7-4c4a-47f7-988e-dfe8cd9d3a12 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:53.037954 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117583b7-4c4a-47f7-988e-dfe8cd9d3a12 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:53.038223 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117583b7-4c4a-47f7-988e-dfe8cd9d3a12 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:53.120248 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117583b7-4c4a-47f7-988e-dfe8cd9d3a12 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:45:53.122158 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117583b7-4c4a-47f7-988e-dfe8cd9d3a12 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:45:53.149808 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:45:53.231428 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 104/206] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Aug 30 15:45:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 30 15:45:53.237730 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bebf52a-597b-4183-b6c3-63a851d6c018 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:45:53.362894 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bebf52a-597b-4183-b6c3-63a851d6c018 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:45:53.398116 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008fbaf0-b753-4626-bbff-7b68f0146461 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:45:53.398789 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008fbaf0-b753-4626-bbff-7b68f0146461 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:53.399059 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008fbaf0-b753-4626-bbff-7b68f0146461 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:53.399335 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008fbaf0-b753-4626-bbff-7b68f0146461 None admin] PATH_INFO: `/v3/projects/2a4e4930fe3247d3b04ad23013176d45` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:53.400471 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008fbaf0-b753-4626-bbff-7b68f0146461 None admin] RBAC: Authorizing `identity:get_project(project_id=2a4e4930fe3247d3b04ad23013176d45)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:45:53.401948 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008fbaf0-b753-4626-bbff-7b68f0146461 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:45:53.403772 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 103/207] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Aug 30 15:45:53 2022] GET /identity/v3/projects/2a4e4930fe3247d3b04ad23013176d45 => generated 298 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:45:53.409805 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117583b7-4c4a-47f7-988e-dfe8cd9d3a12 None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:45:53.526226 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117583b7-4c4a-47f7-988e-dfe8cd9d3a12 None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:45:53.554065 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:45:53.564662 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b13926f-3a6c-4038-b8ae-4290d60e404f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:45:53.565367 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b13926f-3a6c-4038-b8ae-4290d60e404f None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:53.565643 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b13926f-3a6c-4038-b8ae-4290d60e404f None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:53.565956 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b13926f-3a6c-4038-b8ae-4290d60e404f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:53.566691 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b13926f-3a6c-4038-b8ae-4290d60e404f None admin] RBAC: Adding query filter params (role.id=5d293e67c8e747e6910e95cf65e56633, scope.project.id=2a4e4930fe3247d3b04ad23013176d45, user.id=4f46cc84bf04497e86d8762d42fd9a12) {{(pid=81512) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:45:53.566995 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b13926f-3a6c-4038-b8ae-4290d60e404f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:45:53.568505 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b13926f-3a6c-4038-b8ae-4290d60e404f None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:45:53.574797 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 105/208] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Aug 30 15:45:53 2022] GET /identity/v3/role_assignments?user.id=4f46cc84bf04497e86d8762d42fd9a12&scope.project.id=2a4e4930fe3247d3b04ad23013176d45&role.id=5d293e67c8e747e6910e95cf65e56633 => generated 608 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:45:53.757588 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067a72e0-0fe6-4fcf-a408-3c0813e39e6a None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:53.757885 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067a72e0-0fe6-4fcf-a408-3c0813e39e6a None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:53.758064 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067a72e0-0fe6-4fcf-a408-3c0813e39e6a None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:53.759753 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 104/209] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 30 15:45:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 15:45:53.764656 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19abd6b-d7eb-4fe2-a5c2-1037d932ef79 None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:53.764944 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19abd6b-d7eb-4fe2-a5c2-1037d932ef79 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:53.765312 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19abd6b-d7eb-4fe2-a5c2-1037d932ef79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:53.787810 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-a19abd6b-d7eb-4fe2-a5c2-1037d932ef79 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:45:53.789021 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-a19abd6b-d7eb-4fe2-a5c2-1037d932ef79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:45:53.860415 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 106/210] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 30 15:45:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 15:45:53.967538 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0de3521-acd9-451b-b1d6-2173183feea2 None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:53.967868 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0de3521-acd9-451b-b1d6-2173183feea2 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:53.968079 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0de3521-acd9-451b-b1d6-2173183feea2 None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:53.969630 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 105/211] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 30 15:45:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 15:45:53.975101 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852d8121-e23f-4a63-ad0a-861421d687fe None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:53.975388 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852d8121-e23f-4a63-ad0a-861421d687fe None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:53.975663 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852d8121-e23f-4a63-ad0a-861421d687fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:53.998237 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-852d8121-e23f-4a63-ad0a-861421d687fe None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:45:53.999425 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-852d8121-e23f-4a63-ad0a-861421d687fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:45:54.071324 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e58437c-c485-4916-86d0-d6f2308c8fa6 None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:54.071929 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e58437c-c485-4916-86d0-d6f2308c8fa6 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:54.072394 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e58437c-c485-4916-86d0-d6f2308c8fa6 None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:54.074255 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 106/212] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 30 15:45:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 15:45:54.080790 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4479eb40-67a2-4c94-a6fc-8e3a2a03085a None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:54.081413 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4479eb40-67a2-4c94-a6fc-8e3a2a03085a None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:54.081885 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4479eb40-67a2-4c94-a6fc-8e3a2a03085a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:54.084704 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 107/213] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 30 15:45:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 15:45:54.101377 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-4479eb40-67a2-4c94-a6fc-8e3a2a03085a None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:45:54.102875 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-4479eb40-67a2-4c94-a6fc-8e3a2a03085a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:45:54.134630 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:45:54.149468 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852d8121-e23f-4a63-ad0a-861421d687fe None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:45:54.179969 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 107/214] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 30 15:45:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 15:45:54.209022 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:45:54.259130 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852d8121-e23f-4a63-ad0a-861421d687fe None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:45:54.293482 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56933405-e718-4173-90e4-2cff21c8f167 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:45:54.294117 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56933405-e718-4173-90e4-2cff21c8f167 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:54.294396 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56933405-e718-4173-90e4-2cff21c8f167 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:54.294671 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56933405-e718-4173-90e4-2cff21c8f167 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:54.304952 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56933405-e718-4173-90e4-2cff21c8f167 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 15:45:54.309839 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application [None req-56933405-e718-4173-90e4-2cff21c8f167 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 30 15:45:54.309839 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:45:54.309839 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:45:54.309839 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:45:54.309839 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:45:54.309839 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:45:54.309839 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:45:54.309839 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:45:54.309839 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:45:54.309839 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:45:54.309839 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:45:54.309839 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:45:54.309839 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 15:45:54.309839 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 15:45:54.309839 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 15:45:54.309839 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 15:45:54.310701 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 15:45:54.310701 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 15:45:54.310701 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 15:45:54.310701 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 15:45:54.310701 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:45:54.310701 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:45:54.310701 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:45:54.310701 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:45:54.310701 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:45:54.310701 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:45:54.310701 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:45:54.310701 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application with Lock( Aug 30 15:45:54.310701 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:45:54.310701 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:45:54.310701 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:45:54.310701 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:45:54.311531 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:45:54.311531 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:45:54.311531 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:45:54.311531 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:45:54.311531 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 15:45:54.311531 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 15:45:54.311531 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 15:45:54.311531 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 15:45:54.311531 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 15:45:54.311531 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 15:45:54.311531 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 30 15:45:54.311531 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Aug 30 15:45:54.311531 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 108/215] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Aug 30 15:45:54 2022] GET /identity/v3/roles/member => generated 84 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 15:45:54.317119 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4479eb40-67a2-4c94-a6fc-8e3a2a03085a None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:45:54.373632 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56933405-e718-4173-90e4-2cff21c8f167 None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:45:54.422948 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4479eb40-67a2-4c94-a6fc-8e3a2a03085a None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:45:54.460347 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895b2f85-540e-4937-8982-b611f09b02c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:45:54.461368 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895b2f85-540e-4937-8982-b611f09b02c9 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:54.461844 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895b2f85-540e-4937-8982-b611f09b02c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:54.462309 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895b2f85-540e-4937-8982-b611f09b02c9 None admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:54.463284 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895b2f85-540e-4937-8982-b611f09b02c9 None admin] RBAC: Adding query filter params (name=member) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:45:54.463799 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895b2f85-540e-4937-8982-b611f09b02c9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:45:54.465509 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895b2f85-540e-4937-8982-b611f09b02c9 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:45:54.474809 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 108/216] 10.222.0.20 () {56 vars in 1101 bytes} [Tue Aug 30 15:45:54 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 15:45:54.475954 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895b2f85-540e-4937-8982-b611f09b02c9 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:45:54.491520 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56933405-e718-4173-90e4-2cff21c8f167 None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:45:54.505718 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:45:54.508191 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:45:54.528841 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad252258-b33d-4612-899c-be653c278821 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:45:54.529497 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad252258-b33d-4612-899c-be653c278821 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:54.529763 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad252258-b33d-4612-899c-be653c278821 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:54.530028 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad252258-b33d-4612-899c-be653c278821 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:54.540762 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad252258-b33d-4612-899c-be653c278821 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 15:45:54.545576 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application [None req-ad252258-b33d-4612-899c-be653c278821 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 15:45:54.545576 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:45:54.545576 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:45:54.545576 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:45:54.545576 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:45:54.545576 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:45:54.545576 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:45:54.545576 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:45:54.545576 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:45:54.545576 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:45:54.545576 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:45:54.545576 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:45:54.545576 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 15:45:54.545576 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 15:45:54.545576 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 15:45:54.545576 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 15:45:54.546494 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 15:45:54.546494 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 15:45:54.546494 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 15:45:54.546494 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 15:45:54.546494 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:45:54.546494 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:45:54.546494 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:45:54.546494 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:45:54.546494 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:45:54.546494 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:45:54.546494 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:45:54.546494 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application with Lock( Aug 30 15:45:54.546494 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:45:54.546494 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:45:54.546494 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:45:54.546494 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:45:54.547331 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:45:54.547331 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:45:54.547331 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:45:54.547331 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:45:54.547331 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 15:45:54.547331 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 15:45:54.547331 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 15:45:54.547331 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 15:45:54.547331 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 15:45:54.547331 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 15:45:54.547331 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 15:45:54.547331 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Aug 30 15:45:54.547331 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 109/217] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Aug 30 15:45:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 15:45:54.547991 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad252258-b33d-4612-899c-be653c278821 None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:45:54.590368 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895b2f85-540e-4937-8982-b611f09b02c9 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:45:54.626573 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883da3a7-8dcd-4655-b47e-879701830a67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:45:54.627113 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883da3a7-8dcd-4655-b47e-879701830a67 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:54.627386 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883da3a7-8dcd-4655-b47e-879701830a67 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:54.627650 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883da3a7-8dcd-4655-b47e-879701830a67 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:54.630454 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883da3a7-8dcd-4655-b47e-879701830a67 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:45:54.632503 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883da3a7-8dcd-4655-b47e-879701830a67 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:45:54.634485 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 109/218] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 30 15:45:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:45:54.635432 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883da3a7-8dcd-4655-b47e-879701830a67 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:45:54.654173 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad252258-b33d-4612-899c-be653c278821 None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:45:54.690942 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e9769b-53c2-4f99-ba1f-cb516760462c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:45:54.691571 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e9769b-53c2-4f99-ba1f-cb516760462c None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:54.691841 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e9769b-53c2-4f99-ba1f-cb516760462c None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:54.692099 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e9769b-53c2-4f99-ba1f-cb516760462c None admin] PATH_INFO: `/v3/users/470f03f3d02440ad9ffb32c7b79e1743` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:54.705651 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e9769b-53c2-4f99-ba1f-cb516760462c None admin] RBAC: Authorizing `identity:get_user(user_id=470f03f3d02440ad9ffb32c7b79e1743)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:45:54.707144 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e9769b-53c2-4f99-ba1f-cb516760462c None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:45:54.708959 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 110/219] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Aug 30 15:45:54 2022] GET /identity/v3/users/470f03f3d02440ad9ffb32c7b79e1743 => generated 297 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:45:54.709561 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e9769b-53c2-4f99-ba1f-cb516760462c None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:45:54.739851 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:45:54.746015 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:45:54.757164 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883da3a7-8dcd-4655-b47e-879701830a67 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:45:54.791958 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ebd815-1277-4949-8a0a-a0c9fb9e14e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:45:54.792567 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ebd815-1277-4949-8a0a-a0c9fb9e14e4 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:54.792838 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ebd815-1277-4949-8a0a-a0c9fb9e14e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:54.793107 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ebd815-1277-4949-8a0a-a0c9fb9e14e4 None admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:54.793825 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ebd815-1277-4949-8a0a-a0c9fb9e14e4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:45:54.794123 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ebd815-1277-4949-8a0a-a0c9fb9e14e4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:45:54.795508 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ebd815-1277-4949-8a0a-a0c9fb9e14e4 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:45:54.804413 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 110/220] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Aug 30 15:45:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:45:54.805466 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ebd815-1277-4949-8a0a-a0c9fb9e14e4 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:45:54.817794 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e9769b-53c2-4f99-ba1f-cb516760462c None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:45:54.852472 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd5c274-3068-4b16-b8aa-151dbed31daf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:45:54.853074 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd5c274-3068-4b16-b8aa-151dbed31daf None admin] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:54.853436 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd5c274-3068-4b16-b8aa-151dbed31daf None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:54.853665 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd5c274-3068-4b16-b8aa-151dbed31daf None admin] PATH_INFO: `/v3/users` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:54.854344 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd5c274-3068-4b16-b8aa-151dbed31daf None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:45:54.855753 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd5c274-3068-4b16-b8aa-151dbed31daf None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:45:54.880691 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 111/221] 10.222.0.20 () {60 vars in 1132 bytes} [Tue Aug 30 15:45:54 2022] POST /identity/v3/users => generated 297 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 15:45:54.881426 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd5c274-3068-4b16-b8aa-151dbed31daf None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:45:54.921980 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ebd815-1277-4949-8a0a-a0c9fb9e14e4 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:45:54.963726 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302fcf3b-9271-4812-968f-98ad836b2d39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:45:54.964455 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302fcf3b-9271-4812-968f-98ad836b2d39 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:54.964739 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302fcf3b-9271-4812-968f-98ad836b2d39 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:54.965015 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302fcf3b-9271-4812-968f-98ad836b2d39 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:54.981969 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:45:55.000631 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd5c274-3068-4b16-b8aa-151dbed31daf None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:45:55.001167 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:45:55.039391 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217a17ef-7b37-4efa-88ea-fb2b4a220b4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:45:55.040038 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217a17ef-7b37-4efa-88ea-fb2b4a220b4e None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:55.040478 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217a17ef-7b37-4efa-88ea-fb2b4a220b4e None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:55.040623 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217a17ef-7b37-4efa-88ea-fb2b4a220b4e None admin] PATH_INFO: `/v3/users/bc54674c67dc43898519b1e2d04e9aa3` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:55.041852 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217a17ef-7b37-4efa-88ea-fb2b4a220b4e None admin] RBAC: Authorizing `identity:get_user(user_id=bc54674c67dc43898519b1e2d04e9aa3)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:45:55.043339 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217a17ef-7b37-4efa-88ea-fb2b4a220b4e None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:45:55.044975 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 112/222] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Aug 30 15:45:54 2022] GET /identity/v3/users/bc54674c67dc43898519b1e2d04e9aa3 => generated 283 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:45:55.051267 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217a17ef-7b37-4efa-88ea-fb2b4a220b4e None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:45:55.059010 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302fcf3b-9271-4812-968f-98ad836b2d39 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:45:55.060940 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302fcf3b-9271-4812-968f-98ad836b2d39 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:45:55.167434 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217a17ef-7b37-4efa-88ea-fb2b4a220b4e None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:45:55.190504 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 111/223] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Aug 30 15:45:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 15:45:55.191800 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a83fb43-2352-4ace-85ab-adcc5d688529 None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:55.192094 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a83fb43-2352-4ace-85ab-adcc5d688529 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:55.192363 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a83fb43-2352-4ace-85ab-adcc5d688529 None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:55.198641 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 112/224] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 30 15:45:55 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 15:45:55.199672 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a83fb43-2352-4ace-85ab-adcc5d688529 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:45:55.205696 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bab531-1c3d-4b04-8860-7f4d649236fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:45:55.206334 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bab531-1c3d-4b04-8860-7f4d649236fa None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:55.206608 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bab531-1c3d-4b04-8860-7f4d649236fa None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:55.206873 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bab531-1c3d-4b04-8860-7f4d649236fa None admin] PATH_INFO: `/v3/projects/c3ff9ff008b04f21a0d68698cf06e6dc` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:55.208008 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50bab531-1c3d-4b04-8860-7f4d649236fa None admin] RBAC: Authorizing `identity:get_project(project_id=c3ff9ff008b04f21a0d68698cf06e6dc)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:45:55.209704 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50bab531-1c3d-4b04-8860-7f4d649236fa None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:45:55.211641 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 113/225] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Aug 30 15:45:55 2022] GET /identity/v3/projects/c3ff9ff008b04f21a0d68698cf06e6dc => generated 302 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 15:45:55.212787 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7952f608-d4bc-4806-9ffc-134550e1b285 None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:55.213062 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7952f608-d4bc-4806-9ffc-134550e1b285 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:55.217411 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7952f608-d4bc-4806-9ffc-134550e1b285 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:55.236942 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-7952f608-d4bc-4806-9ffc-134550e1b285 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:45:55.238386 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-7952f608-d4bc-4806-9ffc-134550e1b285 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:45:55.258019 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:45:55.277267 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:45:55.312426 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 114/226] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 30 15:45:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 15:45:55.313128 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7952f608-d4bc-4806-9ffc-134550e1b285 None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:45:55.325616 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a83fb43-2352-4ace-85ab-adcc5d688529 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:45:55.371251 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd4ed9a-d191-4645-9366-9637f1572973 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:45:55.373182 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd4ed9a-d191-4645-9366-9637f1572973 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:55.373494 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd4ed9a-d191-4645-9366-9637f1572973 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:55.373766 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd4ed9a-d191-4645-9366-9637f1572973 None admin] PATH_INFO: `/v3/projects/c3ff9ff008b04f21a0d68698cf06e6dc` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:55.375150 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd4ed9a-d191-4645-9366-9637f1572973 None admin] RBAC: Authorizing `identity:get_project(project_id=c3ff9ff008b04f21a0d68698cf06e6dc)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:45:55.377253 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd4ed9a-d191-4645-9366-9637f1572973 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:45:55.379887 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 113/227] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Aug 30 15:45:55 2022] GET /identity/v3/projects/c3ff9ff008b04f21a0d68698cf06e6dc => generated 302 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:45:55.386459 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd4ed9a-d191-4645-9366-9637f1572973 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:45:55.450337 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7952f608-d4bc-4806-9ffc-134550e1b285 None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:45:55.519664 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a8e069-dfe6-4424-b4a5-3adbbef0b712 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:45:55.520373 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a8e069-dfe6-4424-b4a5-3adbbef0b712 None admin] REQUEST_METHOD: `PUT` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:55.520650 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a8e069-dfe6-4424-b4a5-3adbbef0b712 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:55.520920 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a8e069-dfe6-4424-b4a5-3adbbef0b712 None admin] PATH_INFO: `/v3/projects/c3ff9ff008b04f21a0d68698cf06e6dc/users/bc54674c67dc43898519b1e2d04e9aa3/roles/f265386950a641ac8413b53eda4ae34d` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:55.523017 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a8e069-dfe6-4424-b4a5-3adbbef0b712 None admin] RBAC: Authorizing `identity:create_grant(project_id=c3ff9ff008b04f21a0d68698cf06e6dc, user_id=bc54674c67dc43898519b1e2d04e9aa3, role_id=f265386950a641ac8413b53eda4ae34d)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:45:55.524492 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a8e069-dfe6-4424-b4a5-3adbbef0b712 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:45:55.550156 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd4ed9a-d191-4645-9366-9637f1572973 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:45:55.560011 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 115/228] 10.222.0.20 () {58 vars in 1439 bytes} [Tue Aug 30 15:45:55 2022] PUT /identity/v3/projects/c3ff9ff008b04f21a0d68698cf06e6dc/users/bc54674c67dc43898519b1e2d04e9aa3/roles/f265386950a641ac8413b53eda4ae34d => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:45:55.591579 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:45:55.614046 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeef1993-5a7d-4416-8a6c-b378c3bb9c72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:45:55.614699 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeef1993-5a7d-4416-8a6c-b378c3bb9c72 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:55.614979 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeef1993-5a7d-4416-8a6c-b378c3bb9c72 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:55.615251 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeef1993-5a7d-4416-8a6c-b378c3bb9c72 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:55.616010 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeef1993-5a7d-4416-8a6c-b378c3bb9c72 None admin] RBAC: Adding query filter params (role.id=df8f9f844b1b439181d75a07c1a93418, scope.project.id=c3ff9ff008b04f21a0d68698cf06e6dc, user.id=470f03f3d02440ad9ffb32c7b79e1743) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:45:55.616316 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeef1993-5a7d-4416-8a6c-b378c3bb9c72 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:45:55.617812 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeef1993-5a7d-4416-8a6c-b378c3bb9c72 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:45:55.624892 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 114/229] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Aug 30 15:45:55 2022] GET /identity/v3/role_assignments?user.id=470f03f3d02440ad9ffb32c7b79e1743&scope.project.id=c3ff9ff008b04f21a0d68698cf06e6dc&role.id=df8f9f844b1b439181d75a07c1a93418 => generated 259 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:45:55.671534 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a8e069-dfe6-4424-b4a5-3adbbef0b712 None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:45:55.700754 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:45:55.814573 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a8e069-dfe6-4424-b4a5-3adbbef0b712 None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:45:55.856995 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf23774-aa09-4a5c-bd06-e4d83da4017c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:45:55.857673 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf23774-aa09-4a5c-bd06-e4d83da4017c None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:55.857952 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf23774-aa09-4a5c-bd06-e4d83da4017c None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:55.858226 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf23774-aa09-4a5c-bd06-e4d83da4017c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:55.870246 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf23774-aa09-4a5c-bd06-e4d83da4017c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 15:45:55.875507 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application [None req-aaf23774-aa09-4a5c-bd06-e4d83da4017c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 15:45:55.875507 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:45:55.875507 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:45:55.875507 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:45:55.875507 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:45:55.875507 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:45:55.875507 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:45:55.875507 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:45:55.875507 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:45:55.875507 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:45:55.875507 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:45:55.875507 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:45:55.875507 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 15:45:55.875507 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 15:45:55.875507 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 15:45:55.875507 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 15:45:55.876361 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 15:45:55.876361 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 15:45:55.876361 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 15:45:55.876361 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 15:45:55.876361 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:45:55.876361 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:45:55.876361 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:45:55.876361 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:45:55.876361 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:45:55.876361 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:45:55.876361 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:45:55.876361 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application with Lock( Aug 30 15:45:55.876361 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:45:55.876361 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:45:55.876361 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:45:55.876361 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:45:55.877157 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:45:55.877157 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:45:55.877157 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:45:55.877157 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:45:55.877157 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 15:45:55.877157 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 15:45:55.877157 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 15:45:55.877157 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 15:45:55.877157 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 15:45:55.877157 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 15:45:55.877157 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 15:45:55.877157 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Aug 30 15:45:55.877157 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 116/230] 10.222.0.20 () {56 vars in 1114 bytes} [Tue Aug 30 15:45:55 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 15:45:55.882556 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeef1993-5a7d-4416-8a6c-b378c3bb9c72 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:45:55.991955 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeef1993-5a7d-4416-8a6c-b378c3bb9c72 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:45:56.028699 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fe7dac-e1a6-4a6c-9818-34c7c415104b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:45:56.029345 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fe7dac-e1a6-4a6c-9818-34c7c415104b None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:56.029631 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fe7dac-e1a6-4a6c-9818-34c7c415104b None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:56.029904 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fe7dac-e1a6-4a6c-9818-34c7c415104b None admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:56.030638 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fe7dac-e1a6-4a6c-9818-34c7c415104b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:45:56.030975 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fe7dac-e1a6-4a6c-9818-34c7c415104b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:45:56.032475 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fe7dac-e1a6-4a6c-9818-34c7c415104b None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:45:56.041377 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 115/231] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Aug 30 15:45:55 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:45:56.047247 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf23774-aa09-4a5c-bd06-e4d83da4017c None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:45:56.137770 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:45:56.173333 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf23774-aa09-4a5c-bd06-e4d83da4017c None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:45:56.224812 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7d34d6-3cc4-4a95-9fa5-6c66afbe175a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:45:56.225496 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7d34d6-3cc4-4a95-9fa5-6c66afbe175a None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:56.225770 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7d34d6-3cc4-4a95-9fa5-6c66afbe175a None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:56.226050 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7d34d6-3cc4-4a95-9fa5-6c66afbe175a None admin] PATH_INFO: `/v3/users/bc54674c67dc43898519b1e2d04e9aa3` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:56.227209 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7d34d6-3cc4-4a95-9fa5-6c66afbe175a None admin] RBAC: Authorizing `identity:get_user(user_id=bc54674c67dc43898519b1e2d04e9aa3)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:45:56.228679 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7d34d6-3cc4-4a95-9fa5-6c66afbe175a None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:45:56.230486 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 117/232] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Aug 30 15:45:56 2022] GET /identity/v3/users/bc54674c67dc43898519b1e2d04e9aa3 => generated 283 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:45:56.235948 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fe7dac-e1a6-4a6c-9818-34c7c415104b None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:45:56.303248 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:45:56.362401 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fe7dac-e1a6-4a6c-9818-34c7c415104b None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:45:56.408243 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9557878c-9544-4fa7-b5ae-8fdc8b237165 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:45:56.408883 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9557878c-9544-4fa7-b5ae-8fdc8b237165 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:56.409161 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9557878c-9544-4fa7-b5ae-8fdc8b237165 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:56.409459 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9557878c-9544-4fa7-b5ae-8fdc8b237165 None admin] PATH_INFO: `/v3/projects/c3ff9ff008b04f21a0d68698cf06e6dc` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:56.413969 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9557878c-9544-4fa7-b5ae-8fdc8b237165 None admin] RBAC: Authorizing `identity:get_project(project_id=c3ff9ff008b04f21a0d68698cf06e6dc)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:45:56.415499 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9557878c-9544-4fa7-b5ae-8fdc8b237165 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:45:56.417367 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 116/233] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Aug 30 15:45:56 2022] GET /identity/v3/projects/c3ff9ff008b04f21a0d68698cf06e6dc => generated 302 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:45:56.422988 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7d34d6-3cc4-4a95-9fa5-6c66afbe175a None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:45:56.544879 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7d34d6-3cc4-4a95-9fa5-6c66afbe175a None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:45:56.603948 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cf3c90-3cf7-48b3-a377-9d5283e06fe4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:45:56.604540 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cf3c90-3cf7-48b3-a377-9d5283e06fe4 None admin] REQUEST_METHOD: `PUT` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:56.604819 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cf3c90-3cf7-48b3-a377-9d5283e06fe4 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:56.605089 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cf3c90-3cf7-48b3-a377-9d5283e06fe4 None admin] PATH_INFO: `/v3/projects/c3ff9ff008b04f21a0d68698cf06e6dc/users/bc54674c67dc43898519b1e2d04e9aa3/roles/6b4385c3b449456d848c9abb16766d25` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:56.610940 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8cf3c90-3cf7-48b3-a377-9d5283e06fe4 None admin] RBAC: Authorizing `identity:create_grant(project_id=c3ff9ff008b04f21a0d68698cf06e6dc, user_id=bc54674c67dc43898519b1e2d04e9aa3, role_id=6b4385c3b449456d848c9abb16766d25)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:45:56.612421 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8cf3c90-3cf7-48b3-a377-9d5283e06fe4 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:45:56.639637 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:45:56.648975 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 118/234] 10.222.0.20 () {58 vars in 1439 bytes} [Tue Aug 30 15:45:56 2022] PUT /identity/v3/projects/c3ff9ff008b04f21a0d68698cf06e6dc/users/bc54674c67dc43898519b1e2d04e9aa3/roles/6b4385c3b449456d848c9abb16766d25 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:45:57.153414 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e070b1-04ba-4785-bcb3-89ffde2a6c69 None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:57.154378 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e070b1-04ba-4785-bcb3-89ffde2a6c69 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:57.154882 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e070b1-04ba-4785-bcb3-89ffde2a6c69 None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:57.156793 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 117/235] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 30 15:45:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 15:45:57.162755 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b22b85-f16d-4fdc-beb9-ea2dac06e58e None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:57.163565 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b22b85-f16d-4fdc-beb9-ea2dac06e58e None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:57.164163 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b22b85-f16d-4fdc-beb9-ea2dac06e58e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:57.190435 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-03b22b85-f16d-4fdc-beb9-ea2dac06e58e None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:45:57.191888 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-03b22b85-f16d-4fdc-beb9-ea2dac06e58e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:45:57.286737 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 119/236] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 30 15:45:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 15:45:57.326650 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9642f42e-e98e-4c3d-8dec-ab1a5149e04a None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:57.327277 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9642f42e-e98e-4c3d-8dec-ab1a5149e04a None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:57.327798 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9642f42e-e98e-4c3d-8dec-ab1a5149e04a None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:57.329454 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 118/237] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 30 15:45:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 15:45:57.335232 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bf7ba1-ad47-44fa-a839-6dcdfea4a02e None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:57.335521 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bf7ba1-ad47-44fa-a839-6dcdfea4a02e None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:57.335790 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bf7ba1-ad47-44fa-a839-6dcdfea4a02e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:57.359049 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-27bf7ba1-ad47-44fa-a839-6dcdfea4a02e None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:45:57.360240 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-27bf7ba1-ad47-44fa-a839-6dcdfea4a02e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:45:57.454624 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 120/238] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 30 15:45:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 15:45:57.566248 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9642f42e-e98e-4c3d-8dec-ab1a5149e04a None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:45:57.716461 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9642f42e-e98e-4c3d-8dec-ab1a5149e04a None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:45:57.733373 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:45:57.751858 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c92d699-0182-4d8c-800a-8d244e0c3b5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:45:57.753057 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c92d699-0182-4d8c-800a-8d244e0c3b5f None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:57.753370 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c92d699-0182-4d8c-800a-8d244e0c3b5f None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:57.753637 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c92d699-0182-4d8c-800a-8d244e0c3b5f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:57.764528 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c92d699-0182-4d8c-800a-8d244e0c3b5f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 15:45:57.769642 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-5c92d699-0182-4d8c-800a-8d244e0c3b5f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 30 15:45:57.769642 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:45:57.769642 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:45:57.769642 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:45:57.769642 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:45:57.769642 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:45:57.769642 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:45:57.769642 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:45:57.769642 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:45:57.769642 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:45:57.769642 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:45:57.769642 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:45:57.769642 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 15:45:57.769642 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 15:45:57.769642 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 15:45:57.769642 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 15:45:57.770521 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 15:45:57.770521 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 15:45:57.770521 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 15:45:57.770521 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 15:45:57.770521 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:45:57.770521 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:45:57.770521 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:45:57.770521 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:45:57.770521 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:45:57.770521 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:45:57.770521 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:45:57.770521 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Aug 30 15:45:57.770521 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:45:57.770521 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:45:57.770521 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:45:57.770521 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:45:57.771311 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:45:57.771311 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:45:57.771311 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:45:57.771311 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:45:57.771311 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 15:45:57.771311 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 15:45:57.771311 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 15:45:57.771311 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 15:45:57.771311 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 15:45:57.771311 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 15:45:57.771311 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 30 15:45:57.771311 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Aug 30 15:45:57.771311 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 119/239] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Aug 30 15:45:57 2022] GET /identity/v3/roles/reader => generated 84 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 15:45:57.773851 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bf7ba1-ad47-44fa-a839-6dcdfea4a02e None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:45:57.776817 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c92d699-0182-4d8c-800a-8d244e0c3b5f None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:45:57.893585 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:45:57.955403 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c92d699-0182-4d8c-800a-8d244e0c3b5f None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:45:57.959653 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bf7ba1-ad47-44fa-a839-6dcdfea4a02e None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:45:58.004550 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620f9b63-5ac2-4bd2-9393-ab05bfb34bd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:45:58.005112 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620f9b63-5ac2-4bd2-9393-ab05bfb34bd3 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:58.005450 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620f9b63-5ac2-4bd2-9393-ab05bfb34bd3 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:58.005720 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620f9b63-5ac2-4bd2-9393-ab05bfb34bd3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:58.010944 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620f9b63-5ac2-4bd2-9393-ab05bfb34bd3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:45:58.012422 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620f9b63-5ac2-4bd2-9393-ab05bfb34bd3 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:45:58.014093 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 121/240] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 30 15:45:57 2022] GET /identity/v3/domains/default => generated 199 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 15:45:58.019545 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620f9b63-5ac2-4bd2-9393-ab05bfb34bd3 None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:45:58.041537 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe52e70-4b95-4f4d-ab0b-a5f758666099 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:45:58.042159 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe52e70-4b95-4f4d-ab0b-a5f758666099 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:58.042435 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe52e70-4b95-4f4d-ab0b-a5f758666099 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:58.042702 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe52e70-4b95-4f4d-ab0b-a5f758666099 None admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:58.043382 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe52e70-4b95-4f4d-ab0b-a5f758666099 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:45:58.043681 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe52e70-4b95-4f4d-ab0b-a5f758666099 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:45:58.045074 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe52e70-4b95-4f4d-ab0b-a5f758666099 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:45:58.071748 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 120/241] 10.222.0.20 () {56 vars in 1101 bytes} [Tue Aug 30 15:45:57 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:45:58.086690 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe52e70-4b95-4f4d-ab0b-a5f758666099 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:45:58.156402 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620f9b63-5ac2-4bd2-9393-ab05bfb34bd3 None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:45:58.163014 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:45:58.194302 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:45:58.207596 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce107b15-41a8-4d8f-94b1-e53dbaad1c21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:45:58.208254 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce107b15-41a8-4d8f-94b1-e53dbaad1c21 None admin] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:58.209053 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce107b15-41a8-4d8f-94b1-e53dbaad1c21 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:58.209357 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce107b15-41a8-4d8f-94b1-e53dbaad1c21 None admin] PATH_INFO: `/v3/users` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:58.210327 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce107b15-41a8-4d8f-94b1-e53dbaad1c21 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:45:58.212457 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce107b15-41a8-4d8f-94b1-e53dbaad1c21 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:45:58.227857 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe52e70-4b95-4f4d-ab0b-a5f758666099 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:45:58.242072 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 122/242] 10.222.0.20 () {60 vars in 1132 bytes} [Tue Aug 30 15:45:58 2022] POST /identity/v3/users => generated 293 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 15:45:58.243497 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9800eb6b-b809-4cbd-b8fa-9582dfee59aa None None] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:58.243787 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9800eb6b-b809-4cbd-b8fa-9582dfee59aa None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:58.244060 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9800eb6b-b809-4cbd-b8fa-9582dfee59aa None None] PATH_INFO: `/` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:58.245477 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 123/243] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 30 15:45:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 15:45:58.252020 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9783dcda-939a-427e-aa91-d113973a16ec None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:58.252350 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9783dcda-939a-427e-aa91-d113973a16ec None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:58.252572 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9783dcda-939a-427e-aa91-d113973a16ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:58.272451 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-9783dcda-939a-427e-aa91-d113973a16ec None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:45:58.272939 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2ded49-35fd-4f14-8243-fb4177e2dfd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:45:58.273630 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-9783dcda-939a-427e-aa91-d113973a16ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:45:58.273757 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2ded49-35fd-4f14-8243-fb4177e2dfd1 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:58.273937 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2ded49-35fd-4f14-8243-fb4177e2dfd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:58.274176 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2ded49-35fd-4f14-8243-fb4177e2dfd1 None admin] PATH_INFO: `/v3/users/25ef42035fee4b54b04e59d194820e7b` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:58.290473 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2ded49-35fd-4f14-8243-fb4177e2dfd1 None admin] RBAC: Authorizing `identity:get_user(user_id=25ef42035fee4b54b04e59d194820e7b)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:45:58.291872 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2ded49-35fd-4f14-8243-fb4177e2dfd1 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:45:58.294116 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 121/244] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Aug 30 15:45:58 2022] GET /identity/v3/users/25ef42035fee4b54b04e59d194820e7b => generated 297 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 15:45:58.295527 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebccf4d-a74c-4fb8-bf2c-8cf52fe7a4cd None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:58.295806 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebccf4d-a74c-4fb8-bf2c-8cf52fe7a4cd None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:58.296156 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebccf4d-a74c-4fb8-bf2c-8cf52fe7a4cd None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:58.297357 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 122/245] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 30 15:45:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 15:45:58.302219 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebccf4d-a74c-4fb8-bf2c-8cf52fe7a4cd None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:45:58.355787 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 124/246] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 30 15:45:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 15:45:58.357744 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32482362-1a12-4f38-8e37-37dff970ff7b None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:58.358234 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32482362-1a12-4f38-8e37-37dff970ff7b None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:58.358712 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32482362-1a12-4f38-8e37-37dff970ff7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:58.379366 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-32482362-1a12-4f38-8e37-37dff970ff7b None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:45:58.381259 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-32482362-1a12-4f38-8e37-37dff970ff7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:45:58.411888 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:45:58.419326 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebccf4d-a74c-4fb8-bf2c-8cf52fe7a4cd None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:45:58.455790 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 125/247] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 30 15:45:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 15:45:58.462631 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e51d54-716d-45c3-9f1f-34f5531763e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:45:58.462789 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e51d54-716d-45c3-9f1f-34f5531763e7 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:58.463048 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e51d54-716d-45c3-9f1f-34f5531763e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:58.463296 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e51d54-716d-45c3-9f1f-34f5531763e7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:58.537349 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e51d54-716d-45c3-9f1f-34f5531763e7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:45:58.538766 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e51d54-716d-45c3-9f1f-34f5531763e7 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:45:58.637817 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:45:58.641377 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 123/248] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Aug 30 15:45:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 15:45:58.644057 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32482362-1a12-4f38-8e37-37dff970ff7b None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:45:58.674810 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e51d54-716d-45c3-9f1f-34f5531763e7 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:45:58.695495 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:45:58.751836 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32482362-1a12-4f38-8e37-37dff970ff7b None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:45:58.786849 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3ec52c-ea0d-46d4-8464-4d4fb11965a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:45:58.787916 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3ec52c-ea0d-46d4-8464-4d4fb11965a8 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:58.788121 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3ec52c-ea0d-46d4-8464-4d4fb11965a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:58.788387 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3ec52c-ea0d-46d4-8464-4d4fb11965a8 None admin] PATH_INFO: `/v3/projects/c3ff9ff008b04f21a0d68698cf06e6dc` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:58.790753 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3ec52c-ea0d-46d4-8464-4d4fb11965a8 None admin] RBAC: Authorizing `identity:get_project(project_id=c3ff9ff008b04f21a0d68698cf06e6dc)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:45:58.794731 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e51d54-716d-45c3-9f1f-34f5531763e7 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:45:58.796897 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3ec52c-ea0d-46d4-8464-4d4fb11965a8 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:45:58.798665 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 126/249] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Aug 30 15:45:58 2022] GET /identity/v3/projects/c3ff9ff008b04f21a0d68698cf06e6dc => generated 302 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:45:58.799443 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3ec52c-ea0d-46d4-8464-4d4fb11965a8 None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:45:58.838154 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e70af8-b740-4f08-96b4-65b60d97e224 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:45:58.838871 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e70af8-b740-4f08-96b4-65b60d97e224 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:58.839152 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e70af8-b740-4f08-96b4-65b60d97e224 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:58.839424 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e70af8-b740-4f08-96b4-65b60d97e224 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:58.852066 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e70af8-b740-4f08-96b4-65b60d97e224 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 15:45:58.857081 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-42e70af8-b740-4f08-96b4-65b60d97e224 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 30 15:45:58.857081 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:45:58.857081 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:45:58.857081 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:45:58.857081 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:45:58.857081 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:45:58.857081 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:45:58.857081 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:45:58.857081 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:45:58.857081 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:45:58.857081 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:45:58.857081 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:45:58.857081 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 15:45:58.857081 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 15:45:58.857081 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 15:45:58.857081 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 15:45:58.858026 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 15:45:58.858026 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 15:45:58.858026 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 15:45:58.858026 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 15:45:58.858026 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:45:58.858026 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:45:58.858026 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:45:58.858026 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:45:58.858026 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:45:58.858026 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:45:58.858026 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:45:58.858026 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Aug 30 15:45:58.858026 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:45:58.858026 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:45:58.858026 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:45:58.858026 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:45:58.858895 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:45:58.858895 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:45:58.858895 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:45:58.858895 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:45:58.858895 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 15:45:58.858895 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 15:45:58.858895 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 15:45:58.858895 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 15:45:58.858895 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 15:45:58.858895 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 15:45:58.858895 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 30 15:45:58.858895 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Aug 30 15:45:58.860499 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 124/250] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Aug 30 15:45:58 2022] GET /identity/v3/roles/member => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 15:45:58.860499 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e70af8-b740-4f08-96b4-65b60d97e224 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:45:58.932003 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:45:58.942768 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3ec52c-ea0d-46d4-8464-4d4fb11965a8 None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:45:58.964039 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:45:58.998352 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e70af8-b740-4f08-96b4-65b60d97e224 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:45:58.998550 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe61f63f-450c-4a87-b97f-686b3ae65881 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:45:58.999169 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe61f63f-450c-4a87-b97f-686b3ae65881 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:58.999448 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe61f63f-450c-4a87-b97f-686b3ae65881 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:59.000001 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe61f63f-450c-4a87-b97f-686b3ae65881 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:59.017441 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe61f63f-450c-4a87-b97f-686b3ae65881 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 15:45:59.023607 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application [None req-fe61f63f-450c-4a87-b97f-686b3ae65881 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 15:45:59.023607 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:45:59.023607 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:45:59.023607 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:45:59.023607 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:45:59.023607 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:45:59.023607 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:45:59.023607 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:45:59.023607 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:45:59.023607 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:45:59.023607 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:45:59.023607 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:45:59.023607 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 15:45:59.023607 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 15:45:59.023607 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 15:45:59.023607 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 15:45:59.024513 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 15:45:59.024513 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 15:45:59.024513 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 15:45:59.024513 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 15:45:59.024513 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:45:59.024513 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:45:59.024513 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:45:59.024513 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:45:59.024513 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:45:59.024513 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:45:59.024513 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:45:59.024513 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application with Lock( Aug 30 15:45:59.024513 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:45:59.024513 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:45:59.024513 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:45:59.024513 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:45:59.025322 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:45:59.025322 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:45:59.025322 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:45:59.025322 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:45:59.025322 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 15:45:59.025322 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 15:45:59.025322 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 15:45:59.025322 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 15:45:59.025322 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 15:45:59.025322 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 15:45:59.025322 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 15:45:59.025322 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Aug 30 15:45:59.025926 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 127/251] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Aug 30 15:45:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 15:45:59.026719 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe61f63f-450c-4a87-b97f-686b3ae65881 None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:45:59.039025 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d0fedd-dc04-4fba-a097-aab225591b7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:45:59.039564 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d0fedd-dc04-4fba-a097-aab225591b7b None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:59.039839 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d0fedd-dc04-4fba-a097-aab225591b7b None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:59.040106 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d0fedd-dc04-4fba-a097-aab225591b7b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:59.040845 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d0fedd-dc04-4fba-a097-aab225591b7b None admin] RBAC: Adding query filter params (role.id=5d293e67c8e747e6910e95cf65e56633, scope.project.id=c3ff9ff008b04f21a0d68698cf06e6dc, user.id=25ef42035fee4b54b04e59d194820e7b) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:45:59.041148 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d0fedd-dc04-4fba-a097-aab225591b7b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:45:59.042700 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d0fedd-dc04-4fba-a097-aab225591b7b None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:45:59.048897 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 125/252] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Aug 30 15:45:58 2022] GET /identity/v3/role_assignments?user.id=25ef42035fee4b54b04e59d194820e7b&scope.project.id=c3ff9ff008b04f21a0d68698cf06e6dc&role.id=5d293e67c8e747e6910e95cf65e56633 => generated 259 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:45:59.052735 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d0fedd-dc04-4fba-a097-aab225591b7b None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:45:59.176516 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe61f63f-450c-4a87-b97f-686b3ae65881 None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:45:59.238295 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d0fedd-dc04-4fba-a097-aab225591b7b None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:45:59.261941 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc196cad-794d-4edd-897a-92471a8057d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:45:59.263788 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:45:59.264881 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc196cad-794d-4edd-897a-92471a8057d9 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:59.265887 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc196cad-794d-4edd-897a-92471a8057d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:59.266362 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc196cad-794d-4edd-897a-92471a8057d9 None admin] PATH_INFO: `/v3/roles` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:59.267435 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc196cad-794d-4edd-897a-92471a8057d9 None admin] RBAC: Adding query filter params (name=member) {{(pid=81512) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:45:59.268440 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc196cad-794d-4edd-897a-92471a8057d9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:45:59.270987 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc196cad-794d-4edd-897a-92471a8057d9 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:45:59.286177 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 128/253] 10.222.0.20 () {56 vars in 1101 bytes} [Tue Aug 30 15:45:59 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:45:59.286657 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d530b6f3-f9b9-4e4c-a8dc-02885f0f76bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:45:59.287339 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d530b6f3-f9b9-4e4c-a8dc-02885f0f76bd None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:59.287679 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d530b6f3-f9b9-4e4c-a8dc-02885f0f76bd None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:59.288371 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d2d0d1-94bb-4e41-9783-0414b806baad None None] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:59.288936 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d2d0d1-94bb-4e41-9783-0414b806baad None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:59.296512 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d2d0d1-94bb-4e41-9783-0414b806baad None None] PATH_INFO: `/` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:59.296675 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d530b6f3-f9b9-4e4c-a8dc-02885f0f76bd None admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:59.296675 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d530b6f3-f9b9-4e4c-a8dc-02885f0f76bd None admin] RBAC: Adding query filter params (name=admin) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:45:59.296675 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d530b6f3-f9b9-4e4c-a8dc-02885f0f76bd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:45:59.296675 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d530b6f3-f9b9-4e4c-a8dc-02885f0f76bd None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:45:59.298773 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 129/254] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 30 15:45:59 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 15:45:59.299501 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d2d0d1-94bb-4e41-9783-0414b806baad None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:45:59.308326 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 126/255] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Aug 30 15:45:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:45:59.312814 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77524574-504e-4b3a-abec-096e753a8d50 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:59.314120 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77524574-504e-4b3a-abec-096e753a8d50 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:59.314409 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77524574-504e-4b3a-abec-096e753a8d50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:59.351388 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-77524574-504e-4b3a-abec-096e753a8d50 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:45:59.354745 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-77524574-504e-4b3a-abec-096e753a8d50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:45:59.380235 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:45:59.452492 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d2d0d1-94bb-4e41-9783-0414b806baad None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:45:59.467275 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 127/256] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 30 15:45:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 15:45:59.468193 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77524574-504e-4b3a-abec-096e753a8d50 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:45:59.487042 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743a0ea0-a500-410d-83f7-0a96a955a61d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:45:59.487678 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743a0ea0-a500-410d-83f7-0a96a955a61d None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:59.487952 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743a0ea0-a500-410d-83f7-0a96a955a61d None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:59.488222 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743a0ea0-a500-410d-83f7-0a96a955a61d None admin] PATH_INFO: `/v3/users/470f03f3d02440ad9ffb32c7b79e1743` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:59.489405 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743a0ea0-a500-410d-83f7-0a96a955a61d None admin] RBAC: Authorizing `identity:get_user(user_id=470f03f3d02440ad9ffb32c7b79e1743)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:45:59.490870 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743a0ea0-a500-410d-83f7-0a96a955a61d None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:45:59.492842 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 130/257] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Aug 30 15:45:59 2022] GET /identity/v3/users/470f03f3d02440ad9ffb32c7b79e1743 => generated 297 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:45:59.505464 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743a0ea0-a500-410d-83f7-0a96a955a61d None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:45:59.612840 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743a0ea0-a500-410d-83f7-0a96a955a61d None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:45:59.615152 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:45:59.658862 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd801528-01f1-4460-ad4d-a8b02167f6f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:45:59.659516 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd801528-01f1-4460-ad4d-a8b02167f6f0 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:59.659785 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd801528-01f1-4460-ad4d-a8b02167f6f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:59.660053 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd801528-01f1-4460-ad4d-a8b02167f6f0 None admin] PATH_INFO: `/v3/projects/c3ff9ff008b04f21a0d68698cf06e6dc` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:59.661181 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd801528-01f1-4460-ad4d-a8b02167f6f0 None admin] RBAC: Authorizing `identity:get_project(project_id=c3ff9ff008b04f21a0d68698cf06e6dc)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:45:59.662677 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd801528-01f1-4460-ad4d-a8b02167f6f0 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:45:59.662787 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77524574-504e-4b3a-abec-096e753a8d50 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:45:59.664489 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 131/258] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Aug 30 15:45:59 2022] GET /identity/v3/projects/c3ff9ff008b04f21a0d68698cf06e6dc => generated 302 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 15:45:59.671328 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd801528-01f1-4460-ad4d-a8b02167f6f0 None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:45:59.693587 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:45:59.740634 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0bc56f-8500-4d38-8f37-d0b69e33b423 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:45:59.741405 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0bc56f-8500-4d38-8f37-d0b69e33b423 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:59.741679 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0bc56f-8500-4d38-8f37-d0b69e33b423 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:59.741942 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0bc56f-8500-4d38-8f37-d0b69e33b423 None admin] PATH_INFO: `/v3/users/bc54674c67dc43898519b1e2d04e9aa3` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:59.744320 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0bc56f-8500-4d38-8f37-d0b69e33b423 None admin] RBAC: Authorizing `identity:get_user(user_id=bc54674c67dc43898519b1e2d04e9aa3)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:45:59.745817 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0bc56f-8500-4d38-8f37-d0b69e33b423 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:45:59.747815 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 128/259] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Aug 30 15:45:59 2022] GET /identity/v3/users/bc54674c67dc43898519b1e2d04e9aa3 => generated 283 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:45:59.758530 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0bc56f-8500-4d38-8f37-d0b69e33b423 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:45:59.836362 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd801528-01f1-4460-ad4d-a8b02167f6f0 None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:45:59.890887 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0e83c1-fbe4-4eeb-8988-8816dba51170 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:45:59.891563 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0e83c1-fbe4-4eeb-8988-8816dba51170 None admin] REQUEST_METHOD: `PUT` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:45:59.891842 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0e83c1-fbe4-4eeb-8988-8816dba51170 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:45:59.892112 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0e83c1-fbe4-4eeb-8988-8816dba51170 None admin] PATH_INFO: `/v3/projects/c3ff9ff008b04f21a0d68698cf06e6dc/users/470f03f3d02440ad9ffb32c7b79e1743/roles/df8f9f844b1b439181d75a07c1a93418` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:45:59.894065 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0e83c1-fbe4-4eeb-8988-8816dba51170 None admin] RBAC: Authorizing `identity:create_grant(project_id=c3ff9ff008b04f21a0d68698cf06e6dc, user_id=470f03f3d02440ad9ffb32c7b79e1743, role_id=df8f9f844b1b439181d75a07c1a93418)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:45:59.895539 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0e83c1-fbe4-4eeb-8988-8816dba51170 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:45:59.936751 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 132/260] 10.222.0.20 () {58 vars in 1439 bytes} [Tue Aug 30 15:45:59 2022] PUT /identity/v3/projects/c3ff9ff008b04f21a0d68698cf06e6dc/users/470f03f3d02440ad9ffb32c7b79e1743/roles/df8f9f844b1b439181d75a07c1a93418 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 15:45:59.981964 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0e83c1-fbe4-4eeb-8988-8816dba51170 None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:45:59.983022 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0bc56f-8500-4d38-8f37-d0b69e33b423 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:00.055617 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3043c915-2e6b-42c0-a886-17a3da43327f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:00.056253 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3043c915-2e6b-42c0-a886-17a3da43327f None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:00.056526 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3043c915-2e6b-42c0-a886-17a3da43327f None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:00.056789 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3043c915-2e6b-42c0-a886-17a3da43327f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:00.067195 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:00.185724 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:00.207699 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0e83c1-fbe4-4eeb-8988-8816dba51170 None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:00.230282 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3043c915-2e6b-42c0-a886-17a3da43327f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:00.231752 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3043c915-2e6b-42c0-a886-17a3da43327f None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:00.251686 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef2ec78-21fd-4d1e-9d40-fcc05bd00100 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:00.252887 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef2ec78-21fd-4d1e-9d40-fcc05bd00100 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:00.253171 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef2ec78-21fd-4d1e-9d40-fcc05bd00100 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:00.253470 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef2ec78-21fd-4d1e-9d40-fcc05bd00100 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:00.269968 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef2ec78-21fd-4d1e-9d40-fcc05bd00100 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 15:46:00.276560 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application [None req-3ef2ec78-21fd-4d1e-9d40-fcc05bd00100 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 15:46:00.276560 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:46:00.276560 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:46:00.276560 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:46:00.276560 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:46:00.276560 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:46:00.276560 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:46:00.276560 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:46:00.276560 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:46:00.276560 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:46:00.276560 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:46:00.276560 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:46:00.276560 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 15:46:00.276560 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 15:46:00.276560 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 15:46:00.276560 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 15:46:00.277467 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 15:46:00.277467 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 15:46:00.277467 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 15:46:00.277467 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 15:46:00.277467 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:46:00.277467 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:46:00.277467 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:46:00.277467 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:46:00.277467 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:46:00.277467 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:46:00.277467 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:46:00.277467 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application with Lock( Aug 30 15:46:00.277467 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:46:00.277467 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:46:00.277467 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:46:00.277467 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:46:00.278273 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:46:00.278273 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:46:00.278273 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:46:00.278273 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:46:00.278273 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 15:46:00.278273 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 15:46:00.278273 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 15:46:00.278273 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 15:46:00.278273 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 15:46:00.278273 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 15:46:00.278273 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 15:46:00.278273 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Aug 30 15:46:00.278273 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 133/261] 10.222.0.20 () {56 vars in 1114 bytes} [Tue Aug 30 15:45:59 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 15:46:00.285633 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef2ec78-21fd-4d1e-9d40-fcc05bd00100 None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:00.472137 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 129/262] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Aug 30 15:45:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 714 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 15:46:00.481244 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3043c915-2e6b-42c0-a886-17a3da43327f None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:00.484425 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:00.509288 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef2ec78-21fd-4d1e-9d40-fcc05bd00100 None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:00.592650 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac6f0ae-36b0-4d3c-bea2-bc0d0cd11a43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:00.593709 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac6f0ae-36b0-4d3c-bea2-bc0d0cd11a43 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:00.594187 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac6f0ae-36b0-4d3c-bea2-bc0d0cd11a43 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:00.594701 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac6f0ae-36b0-4d3c-bea2-bc0d0cd11a43 None admin] PATH_INFO: `/v3/roles` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:00.595578 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac6f0ae-36b0-4d3c-bea2-bc0d0cd11a43 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81512) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:46:00.596662 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac6f0ae-36b0-4d3c-bea2-bc0d0cd11a43 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:00.598333 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac6f0ae-36b0-4d3c-bea2-bc0d0cd11a43 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:00.613829 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 134/263] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Aug 30 15:46:00 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 15:46:00.619658 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac6f0ae-36b0-4d3c-bea2-bc0d0cd11a43 None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:00.748145 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:00.799389 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3043c915-2e6b-42c0-a886-17a3da43327f None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:00.867496 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac6f0ae-36b0-4d3c-bea2-bc0d0cd11a43 None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:00.893946 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1f4d0c-6361-458a-a482-15cea02eac37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:00.893946 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1f4d0c-6361-458a-a482-15cea02eac37 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:00.893946 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1f4d0c-6361-458a-a482-15cea02eac37 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:00.893946 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1f4d0c-6361-458a-a482-15cea02eac37 None admin] PATH_INFO: `/v3/projects/c3ff9ff008b04f21a0d68698cf06e6dc` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:00.897934 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1f4d0c-6361-458a-a482-15cea02eac37 None admin] RBAC: Authorizing `identity:get_project(project_id=c3ff9ff008b04f21a0d68698cf06e6dc)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:00.898569 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1f4d0c-6361-458a-a482-15cea02eac37 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:00.906051 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 130/264] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Aug 30 15:46:00 2022] GET /identity/v3/projects/c3ff9ff008b04f21a0d68698cf06e6dc => generated 302 bytes in 428 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 15:46:00.912056 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1f4d0c-6361-458a-a482-15cea02eac37 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:00.974493 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37205790-20fe-447e-a673-78bb9b41a104 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:00.975149 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37205790-20fe-447e-a673-78bb9b41a104 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:00.975426 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37205790-20fe-447e-a673-78bb9b41a104 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:00.975695 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37205790-20fe-447e-a673-78bb9b41a104 None admin] PATH_INFO: `/v3/users/bc54674c67dc43898519b1e2d04e9aa3` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:00.981901 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37205790-20fe-447e-a673-78bb9b41a104 None admin] RBAC: Authorizing `identity:get_user(user_id=bc54674c67dc43898519b1e2d04e9aa3)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:00.983358 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37205790-20fe-447e-a673-78bb9b41a104 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:00.985050 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 135/265] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Aug 30 15:46:00 2022] GET /identity/v3/users/bc54674c67dc43898519b1e2d04e9aa3 => generated 283 bytes in 366 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 15:46:00.990696 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37205790-20fe-447e-a673-78bb9b41a104 None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:01.044735 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:01.116089 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1f4d0c-6361-458a-a482-15cea02eac37 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:01.188671 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d979a5ab-45b0-4038-81f7-2c4ef31c37fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:01.189322 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d979a5ab-45b0-4038-81f7-2c4ef31c37fc None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:01.189599 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d979a5ab-45b0-4038-81f7-2c4ef31c37fc None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:01.189869 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d979a5ab-45b0-4038-81f7-2c4ef31c37fc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:01.190776 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d979a5ab-45b0-4038-81f7-2c4ef31c37fc None admin] RBAC: Adding query filter params (role.id=f265386950a641ac8413b53eda4ae34d, scope.project.id=c3ff9ff008b04f21a0d68698cf06e6dc, user.id=bc54674c67dc43898519b1e2d04e9aa3) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:46:01.191084 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d979a5ab-45b0-4038-81f7-2c4ef31c37fc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:01.192891 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d979a5ab-45b0-4038-81f7-2c4ef31c37fc None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:01.204401 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37205790-20fe-447e-a673-78bb9b41a104 None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:01.209582 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 131/266] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Aug 30 15:46:00 2022] GET /identity/v3/role_assignments?user.id=bc54674c67dc43898519b1e2d04e9aa3&scope.project.id=c3ff9ff008b04f21a0d68698cf06e6dc&role.id=f265386950a641ac8413b53eda4ae34d => generated 608 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:46:01.264773 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82f1666-1693-481d-acf3-0a6e247f97fe None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:01.265081 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82f1666-1693-481d-acf3-0a6e247f97fe None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:01.265404 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82f1666-1693-481d-acf3-0a6e247f97fe None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:01.266879 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 132/267] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 30 15:46:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 15:46:01.272345 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4fba81-cdab-4dd6-9208-956de8e11139 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:01.272673 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4fba81-cdab-4dd6-9208-956de8e11139 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:01.272967 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4fba81-cdab-4dd6-9208-956de8e11139 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:01.280031 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd16748-2316-4853-831b-c6e1ee49dd35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:01.282374 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd16748-2316-4853-831b-c6e1ee49dd35 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:01.282805 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd16748-2316-4853-831b-c6e1ee49dd35 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:01.283223 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd16748-2316-4853-831b-c6e1ee49dd35 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:01.304164 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-7f4fba81-cdab-4dd6-9208-956de8e11139 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:46:01.307359 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-7f4fba81-cdab-4dd6-9208-956de8e11139 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:46:01.320542 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:01.437950 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd16748-2316-4853-831b-c6e1ee49dd35 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:01.439613 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd16748-2316-4853-831b-c6e1ee49dd35 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:01.459514 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 133/268] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 30 15:46:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 15:46:01.460850 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d303998-b2a3-4fc3-ad59-d646436c8420 None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:01.461183 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d303998-b2a3-4fc3-ad59-d646436c8420 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:01.461486 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d303998-b2a3-4fc3-ad59-d646436c8420 None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:01.465065 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 134/269] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 30 15:46:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 15:46:01.468059 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2bded8-e5d2-4891-90ba-008f044ae2cb None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:01.468408 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2bded8-e5d2-4891-90ba-008f044ae2cb None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:01.468678 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2bded8-e5d2-4891-90ba-008f044ae2cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:01.492395 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:01.503176 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-4b2bded8-e5d2-4891-90ba-008f044ae2cb None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:46:01.507006 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-4b2bded8-e5d2-4891-90ba-008f044ae2cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:46:01.613106 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 136/270] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Aug 30 15:46:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 623 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 30 15:46:01.614185 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34017b32-f694-4914-a6a5-ba04aeb31967 None None] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:01.614472 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34017b32-f694-4914-a6a5-ba04aeb31967 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:01.614744 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34017b32-f694-4914-a6a5-ba04aeb31967 None None] PATH_INFO: `/` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:01.616286 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 137/271] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 30 15:46:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 15:46:01.621858 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe6a6fb-228f-4cc5-bb4a-df1492d11959 None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:01.621998 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe6a6fb-228f-4cc5-bb4a-df1492d11959 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:01.622691 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe6a6fb-228f-4cc5-bb4a-df1492d11959 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:01.638412 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 135/272] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 30 15:46:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 15:46:01.639564 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2bded8-e5d2-4891-90ba-008f044ae2cb None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:01.659273 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-5fe6a6fb-228f-4cc5-bb4a-df1492d11959 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:46:01.660462 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-5fe6a6fb-228f-4cc5-bb4a-df1492d11959 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:46:01.776205 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:01.791215 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 138/273] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 30 15:46:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 15:46:01.792107 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe6a6fb-228f-4cc5-bb4a-df1492d11959 None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:01.817622 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2bded8-e5d2-4891-90ba-008f044ae2cb None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:01.852307 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:01.859171 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817b3b56-9dcc-4387-958a-bcf26cb64b32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:01.859821 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817b3b56-9dcc-4387-958a-bcf26cb64b32 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:01.860100 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817b3b56-9dcc-4387-958a-bcf26cb64b32 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:01.860371 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817b3b56-9dcc-4387-958a-bcf26cb64b32 None admin] PATH_INFO: `/v3/projects/c3ff9ff008b04f21a0d68698cf06e6dc` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:01.861661 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817b3b56-9dcc-4387-958a-bcf26cb64b32 None admin] RBAC: Authorizing `identity:get_project(project_id=c3ff9ff008b04f21a0d68698cf06e6dc)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:01.863471 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817b3b56-9dcc-4387-958a-bcf26cb64b32 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:01.865710 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 136/274] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Aug 30 15:46:01 2022] GET /identity/v3/projects/c3ff9ff008b04f21a0d68698cf06e6dc => generated 302 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:46:01.871772 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817b3b56-9dcc-4387-958a-bcf26cb64b32 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:01.964116 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe6a6fb-228f-4cc5-bb4a-df1492d11959 None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:02.026831 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8a27f9-6ab7-4b74-8f2f-5017a8a65571 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:02.027856 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8a27f9-6ab7-4b74-8f2f-5017a8a65571 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:02.028344 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8a27f9-6ab7-4b74-8f2f-5017a8a65571 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:02.028880 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8a27f9-6ab7-4b74-8f2f-5017a8a65571 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:02.030217 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8a27f9-6ab7-4b74-8f2f-5017a8a65571 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:02.032110 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8a27f9-6ab7-4b74-8f2f-5017a8a65571 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:02.034220 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 139/275] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 30 15:46:01 2022] GET /identity/v3/domains/default => generated 199 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:46:02.034343 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817b3b56-9dcc-4387-958a-bcf26cb64b32 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:02.035361 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8a27f9-6ab7-4b74-8f2f-5017a8a65571 None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:02.079903 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49bf153-b576-492c-9c80-a352d171cfba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:02.080640 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49bf153-b576-492c-9c80-a352d171cfba None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:02.080918 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49bf153-b576-492c-9c80-a352d171cfba None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:02.081186 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49bf153-b576-492c-9c80-a352d171cfba None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:02.082423 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49bf153-b576-492c-9c80-a352d171cfba None admin] RBAC: Adding query filter params (role.id=6b4385c3b449456d848c9abb16766d25, scope.project.id=c3ff9ff008b04f21a0d68698cf06e6dc, user.id=bc54674c67dc43898519b1e2d04e9aa3) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:46:02.082739 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49bf153-b576-492c-9c80-a352d171cfba None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:02.084512 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49bf153-b576-492c-9c80-a352d171cfba None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:02.097244 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 137/276] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Aug 30 15:46:01 2022] GET /identity/v3/role_assignments?user.id=bc54674c67dc43898519b1e2d04e9aa3&scope.project.id=c3ff9ff008b04f21a0d68698cf06e6dc&role.id=6b4385c3b449456d848c9abb16766d25 => generated 608 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:46:02.097244 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49bf153-b576-492c-9c80-a352d171cfba None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:02.098260 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:02.152464 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8a27f9-6ab7-4b74-8f2f-5017a8a65571 None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:02.172578 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:02.192434 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cea68f-0924-4901-baae-d7df3a9fd9aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:02.193136 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cea68f-0924-4901-baae-d7df3a9fd9aa None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:02.193492 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cea68f-0924-4901-baae-d7df3a9fd9aa None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:02.193758 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cea68f-0924-4901-baae-d7df3a9fd9aa None admin] PATH_INFO: `/v3/roles/member` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:02.206025 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48cea68f-0924-4901-baae-d7df3a9fd9aa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 15:46:02.211213 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application [None req-48cea68f-0924-4901-baae-d7df3a9fd9aa None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 30 15:46:02.211213 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:46:02.211213 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:46:02.211213 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:46:02.211213 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:46:02.211213 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:46:02.211213 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:46:02.211213 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:46:02.211213 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:46:02.211213 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:46:02.211213 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:46:02.211213 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:46:02.211213 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 15:46:02.211213 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 15:46:02.211213 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 15:46:02.211213 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 15:46:02.212379 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 15:46:02.212379 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 15:46:02.212379 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 15:46:02.212379 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 15:46:02.212379 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:46:02.212379 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:46:02.212379 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:46:02.212379 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:46:02.212379 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:46:02.212379 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:46:02.212379 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:46:02.212379 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application with Lock( Aug 30 15:46:02.212379 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:46:02.212379 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:46:02.212379 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:46:02.212379 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:46:02.213208 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:46:02.213208 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:46:02.213208 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:46:02.213208 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:46:02.213208 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 15:46:02.213208 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 15:46:02.213208 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 15:46:02.213208 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 15:46:02.213208 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 15:46:02.213208 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 15:46:02.213208 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 30 15:46:02.213208 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Aug 30 15:46:02.213208 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 140/277] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Aug 30 15:46:02 2022] GET /identity/v3/roles/member => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 15:46:02.213208 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cea68f-0924-4901-baae-d7df3a9fd9aa None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:02.224832 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49bf153-b576-492c-9c80-a352d171cfba None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:02.263826 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b002dfc-b2af-4e39-b611-6f460a5d33db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:02.264455 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b002dfc-b2af-4e39-b611-6f460a5d33db None admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:02.264735 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b002dfc-b2af-4e39-b611-6f460a5d33db None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:02.265002 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b002dfc-b2af-4e39-b611-6f460a5d33db None admin] PATH_INFO: `/v3/users` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:02.266942 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b002dfc-b2af-4e39-b611-6f460a5d33db None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:02.267336 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b002dfc-b2af-4e39-b611-6f460a5d33db None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:02.294305 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 138/278] 10.222.0.20 () {60 vars in 1132 bytes} [Tue Aug 30 15:46:02 2022] POST /identity/v3/users => generated 293 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 15:46:02.295368 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b002dfc-b2af-4e39-b611-6f460a5d33db None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:02.341669 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cea68f-0924-4901-baae-d7df3a9fd9aa None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:02.391607 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:02.395324 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f7bfe8-8844-43cc-830b-42842a1547cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:02.397128 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f7bfe8-8844-43cc-830b-42842a1547cc None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:02.397621 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f7bfe8-8844-43cc-830b-42842a1547cc None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:02.397906 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f7bfe8-8844-43cc-830b-42842a1547cc None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:02.414451 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f7bfe8-8844-43cc-830b-42842a1547cc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 15:46:02.420194 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application [None req-61f7bfe8-8844-43cc-830b-42842a1547cc None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 30 15:46:02.420194 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:46:02.420194 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:46:02.420194 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:46:02.420194 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:46:02.420194 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:46:02.420194 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:46:02.420194 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:46:02.420194 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:46:02.420194 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:46:02.420194 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:46:02.420194 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:46:02.420194 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 15:46:02.420194 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 15:46:02.420194 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 15:46:02.420194 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 15:46:02.421138 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 15:46:02.421138 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 15:46:02.421138 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 15:46:02.421138 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 15:46:02.421138 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:46:02.421138 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:46:02.421138 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:46:02.421138 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:46:02.421138 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:46:02.421138 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:46:02.421138 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:46:02.421138 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application with Lock( Aug 30 15:46:02.421138 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:46:02.421138 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:46:02.421138 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:46:02.421138 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:46:02.422089 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:46:02.422089 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:46:02.422089 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:46:02.422089 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:46:02.422089 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 15:46:02.422089 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 15:46:02.422089 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 15:46:02.422089 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 15:46:02.422089 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 15:46:02.422089 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 15:46:02.422089 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 30 15:46:02.422089 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Aug 30 15:46:02.422089 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 141/279] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Aug 30 15:46:02 2022] GET /identity/v3/roles/reader => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 15:46:02.430766 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f7bfe8-8844-43cc-830b-42842a1547cc None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:02.462706 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b002dfc-b2af-4e39-b611-6f460a5d33db None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:02.479465 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:02.507075 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec221de9-de3f-4b20-851e-d3a84c72ba14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:02.507701 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec221de9-de3f-4b20-851e-d3a84c72ba14 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:02.508016 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec221de9-de3f-4b20-851e-d3a84c72ba14 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:02.508275 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec221de9-de3f-4b20-851e-d3a84c72ba14 None admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:02.508954 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec221de9-de3f-4b20-851e-d3a84c72ba14 None admin] RBAC: Adding query filter params (name=member) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:46:02.509299 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec221de9-de3f-4b20-851e-d3a84c72ba14 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:02.511179 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec221de9-de3f-4b20-851e-d3a84c72ba14 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:02.523141 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 139/280] 10.222.0.20 () {56 vars in 1101 bytes} [Tue Aug 30 15:46:02 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:46:02.529064 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec221de9-de3f-4b20-851e-d3a84c72ba14 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:02.564912 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f7bfe8-8844-43cc-830b-42842a1547cc None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:02.608731 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbeb165b-a84a-4c59-8ad9-085b6a424a12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:02.609929 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbeb165b-a84a-4c59-8ad9-085b6a424a12 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:02.610211 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbeb165b-a84a-4c59-8ad9-085b6a424a12 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:02.610483 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbeb165b-a84a-4c59-8ad9-085b6a424a12 None admin] PATH_INFO: `/v3/roles` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:02.611332 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbeb165b-a84a-4c59-8ad9-085b6a424a12 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81512) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:46:02.611639 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbeb165b-a84a-4c59-8ad9-085b6a424a12 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:02.613592 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbeb165b-a84a-4c59-8ad9-085b6a424a12 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:02.624310 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 142/281] 10.222.0.20 () {56 vars in 1101 bytes} [Tue Aug 30 15:46:02 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:46:02.635832 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbeb165b-a84a-4c59-8ad9-085b6a424a12 None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:02.667434 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec221de9-de3f-4b20-851e-d3a84c72ba14 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:02.704436 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:02.719049 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3d99f6-7cc1-42ce-9600-ea65ad165165 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:02.719992 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3d99f6-7cc1-42ce-9600-ea65ad165165 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:02.720431 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3d99f6-7cc1-42ce-9600-ea65ad165165 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:02.720852 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3d99f6-7cc1-42ce-9600-ea65ad165165 None admin] PATH_INFO: `/v3/users/1c5d0a4e865d4bc38d4bfa8256548d3c` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:02.739076 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3d99f6-7cc1-42ce-9600-ea65ad165165 None admin] RBAC: Authorizing `identity:get_user(user_id=1c5d0a4e865d4bc38d4bfa8256548d3c)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:02.739485 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3d99f6-7cc1-42ce-9600-ea65ad165165 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:02.741494 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 140/282] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Aug 30 15:46:02 2022] GET /identity/v3/users/1c5d0a4e865d4bc38d4bfa8256548d3c => generated 293 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 15:46:02.742640 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ae4021-fc46-44e3-b5fb-cb01b5c6cff5 None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:02.742975 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ae4021-fc46-44e3-b5fb-cb01b5c6cff5 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:02.743249 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ae4021-fc46-44e3-b5fb-cb01b5c6cff5 None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:02.744568 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 141/283] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 30 15:46:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 15:46:02.749601 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ae4021-fc46-44e3-b5fb-cb01b5c6cff5 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:02.762569 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:02.766110 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbeb165b-a84a-4c59-8ad9-085b6a424a12 None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:02.814361 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7d2350-4d35-4b9d-a0da-becc4d72a240 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:02.815015 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7d2350-4d35-4b9d-a0da-becc4d72a240 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:02.815291 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7d2350-4d35-4b9d-a0da-becc4d72a240 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:02.815561 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7d2350-4d35-4b9d-a0da-becc4d72a240 None admin] PATH_INFO: `/v3/users/25ef42035fee4b54b04e59d194820e7b` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:02.818348 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7d2350-4d35-4b9d-a0da-becc4d72a240 None admin] RBAC: Authorizing `identity:get_user(user_id=25ef42035fee4b54b04e59d194820e7b)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:02.820475 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7d2350-4d35-4b9d-a0da-becc4d72a240 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:02.822936 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 143/284] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Aug 30 15:46:02 2022] GET /identity/v3/users/25ef42035fee4b54b04e59d194820e7b => generated 297 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 15:46:02.824249 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8dfe55-194c-40fd-bb82-c71cee0e733f None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:02.824688 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8dfe55-194c-40fd-bb82-c71cee0e733f None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:02.825080 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8dfe55-194c-40fd-bb82-c71cee0e733f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:02.848050 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-1d8dfe55-194c-40fd-bb82-c71cee0e733f None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:46:02.849783 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-1d8dfe55-194c-40fd-bb82-c71cee0e733f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:46:02.868613 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ae4021-fc46-44e3-b5fb-cb01b5c6cff5 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:02.907757 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541bc096-cf1e-4028-89eb-ac9db0a90434 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:02.908613 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541bc096-cf1e-4028-89eb-ac9db0a90434 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:02.908992 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541bc096-cf1e-4028-89eb-ac9db0a90434 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:02.909344 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541bc096-cf1e-4028-89eb-ac9db0a90434 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:02.910249 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541bc096-cf1e-4028-89eb-ac9db0a90434 None admin] RBAC: Adding query filter params (role.id=df8f9f844b1b439181d75a07c1a93418, user.id=1c5d0a4e865d4bc38d4bfa8256548d3c, scope.system=all) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:46:02.910654 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541bc096-cf1e-4028-89eb-ac9db0a90434 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:02.912553 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541bc096-cf1e-4028-89eb-ac9db0a90434 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:02.925339 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 142/285] 10.222.0.20 () {56 vars in 1308 bytes} [Tue Aug 30 15:46:02 2022] GET /identity/v3/role_assignments?user.id=1c5d0a4e865d4bc38d4bfa8256548d3c&scope.system=all&role.id=df8f9f844b1b439181d75a07c1a93418 => generated 226 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 15:46:02.925339 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541bc096-cf1e-4028-89eb-ac9db0a90434 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:02.957143 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 144/286] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 30 15:46:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 15:46:03.024488 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:03.111721 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541bc096-cf1e-4028-89eb-ac9db0a90434 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:03.146701 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46219e22-d058-4379-a5a6-2b8499b1d84a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:03.147348 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46219e22-d058-4379-a5a6-2b8499b1d84a None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:03.147622 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46219e22-d058-4379-a5a6-2b8499b1d84a None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:03.147892 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46219e22-d058-4379-a5a6-2b8499b1d84a None admin] PATH_INFO: `/v3/projects/c3ff9ff008b04f21a0d68698cf06e6dc` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:03.149015 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46219e22-d058-4379-a5a6-2b8499b1d84a None admin] RBAC: Authorizing `identity:get_project(project_id=c3ff9ff008b04f21a0d68698cf06e6dc)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:03.150509 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46219e22-d058-4379-a5a6-2b8499b1d84a None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:03.152632 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 143/287] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Aug 30 15:46:02 2022] GET /identity/v3/projects/c3ff9ff008b04f21a0d68698cf06e6dc => generated 302 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:46:03.158071 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8dfe55-194c-40fd-bb82-c71cee0e733f None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:03.251619 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46219e22-d058-4379-a5a6-2b8499b1d84a None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:03.275766 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:03.279750 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8dfe55-194c-40fd-bb82-c71cee0e733f None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:03.315793 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b31acb-c904-423d-aadc-388fec2203b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:03.316460 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b31acb-c904-423d-aadc-388fec2203b6 None admin] REQUEST_METHOD: `PUT` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:03.316738 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b31acb-c904-423d-aadc-388fec2203b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:03.317007 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b31acb-c904-423d-aadc-388fec2203b6 None admin] PATH_INFO: `/v3/projects/c3ff9ff008b04f21a0d68698cf06e6dc/users/25ef42035fee4b54b04e59d194820e7b/roles/5d293e67c8e747e6910e95cf65e56633` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:03.319018 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b31acb-c904-423d-aadc-388fec2203b6 None admin] RBAC: Authorizing `identity:create_grant(project_id=c3ff9ff008b04f21a0d68698cf06e6dc, user_id=25ef42035fee4b54b04e59d194820e7b, role_id=5d293e67c8e747e6910e95cf65e56633)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:03.320478 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b31acb-c904-423d-aadc-388fec2203b6 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:03.338073 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 145/288] 10.222.0.20 () {58 vars in 1439 bytes} [Tue Aug 30 15:46:03 2022] PUT /identity/v3/projects/c3ff9ff008b04f21a0d68698cf06e6dc/users/25ef42035fee4b54b04e59d194820e7b/roles/5d293e67c8e747e6910e95cf65e56633 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:46:03.397673 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46219e22-d058-4379-a5a6-2b8499b1d84a None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:03.412874 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:03.443156 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534708de-a9be-48cb-88ea-9f444f624270 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:03.443816 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534708de-a9be-48cb-88ea-9f444f624270 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:03.444155 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534708de-a9be-48cb-88ea-9f444f624270 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:03.444480 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534708de-a9be-48cb-88ea-9f444f624270 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:03.463913 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534708de-a9be-48cb-88ea-9f444f624270 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 15:46:03.469242 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-534708de-a9be-48cb-88ea-9f444f624270 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 30 15:46:03.469242 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:46:03.469242 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:46:03.469242 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:46:03.469242 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:46:03.469242 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:46:03.469242 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:46:03.469242 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:46:03.469242 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:46:03.469242 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:46:03.469242 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:46:03.469242 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:46:03.469242 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 15:46:03.469242 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 15:46:03.469242 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 15:46:03.469242 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 15:46:03.470178 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 15:46:03.470178 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 15:46:03.470178 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 15:46:03.470178 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 15:46:03.470178 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:46:03.470178 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:46:03.470178 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:46:03.470178 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:46:03.470178 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:46:03.470178 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:46:03.470178 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:46:03.470178 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Aug 30 15:46:03.470178 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:46:03.470178 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:46:03.470178 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:46:03.470178 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:46:03.470965 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:46:03.470965 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:46:03.470965 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:46:03.470965 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:46:03.470965 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 15:46:03.470965 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 15:46:03.470965 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 15:46:03.470965 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 15:46:03.470965 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 15:46:03.470965 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 15:46:03.470965 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 30 15:46:03.470965 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Aug 30 15:46:03.471569 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 144/289] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Aug 30 15:46:03 2022] GET /identity/v3/roles/member => generated 84 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 15:46:03.476458 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b31acb-c904-423d-aadc-388fec2203b6 None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:03.616968 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b31acb-c904-423d-aadc-388fec2203b6 None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:03.666501 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b32648-c88d-4392-90be-c66777f01a87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:03.667117 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b32648-c88d-4392-90be-c66777f01a87 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:03.667395 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b32648-c88d-4392-90be-c66777f01a87 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:03.667662 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b32648-c88d-4392-90be-c66777f01a87 None admin] PATH_INFO: `/v3/roles` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:03.668332 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b32648-c88d-4392-90be-c66777f01a87 None admin] RBAC: Adding query filter params (name=member) {{(pid=81512) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:46:03.668630 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b32648-c88d-4392-90be-c66777f01a87 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:03.670030 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b32648-c88d-4392-90be-c66777f01a87 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:03.681198 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 146/290] 10.222.0.20 () {56 vars in 1101 bytes} [Tue Aug 30 15:46:03 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:46:03.686893 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534708de-a9be-48cb-88ea-9f444f624270 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:03.831163 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534708de-a9be-48cb-88ea-9f444f624270 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:03.879105 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c667c059-7cf8-4dcb-ae09-e11fe61ebd4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:03.879741 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c667c059-7cf8-4dcb-ae09-e11fe61ebd4e None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:03.880023 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c667c059-7cf8-4dcb-ae09-e11fe61ebd4e None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:03.880294 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c667c059-7cf8-4dcb-ae09-e11fe61ebd4e None admin] PATH_INFO: `/v3/users/470f03f3d02440ad9ffb32c7b79e1743` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:03.881536 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c667c059-7cf8-4dcb-ae09-e11fe61ebd4e None admin] RBAC: Authorizing `identity:get_user(user_id=470f03f3d02440ad9ffb32c7b79e1743)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:03.883011 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c667c059-7cf8-4dcb-ae09-e11fe61ebd4e None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:03.884780 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 145/291] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Aug 30 15:46:03 2022] GET /identity/v3/users/470f03f3d02440ad9ffb32c7b79e1743 => generated 297 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 15:46:03.890525 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b32648-c88d-4392-90be-c66777f01a87 None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:03.938260 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:04.029019 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b32648-c88d-4392-90be-c66777f01a87 None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:04.068800 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fd50ed-d84c-4a60-be87-87e825c23ae5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:04.069447 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fd50ed-d84c-4a60-be87-87e825c23ae5 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:04.069723 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fd50ed-d84c-4a60-be87-87e825c23ae5 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:04.069990 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fd50ed-d84c-4a60-be87-87e825c23ae5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:04.156506 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46fd50ed-d84c-4a60-be87-87e825c23ae5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:04.162524 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46fd50ed-d84c-4a60-be87-87e825c23ae5 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:04.187968 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:04.288764 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 147/292] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Aug 30 15:46:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 15:46:04.295392 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c667c059-7cf8-4dcb-ae09-e11fe61ebd4e None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:04.366643 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:04.442699 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c667c059-7cf8-4dcb-ae09-e11fe61ebd4e None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:04.490024 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1b41dd-094e-4a9c-ad19-f1ac77bdb6dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:04.490668 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1b41dd-094e-4a9c-ad19-f1ac77bdb6dd None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:04.490943 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1b41dd-094e-4a9c-ad19-f1ac77bdb6dd None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:04.491211 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1b41dd-094e-4a9c-ad19-f1ac77bdb6dd None admin] PATH_INFO: `/v3/projects/c3ff9ff008b04f21a0d68698cf06e6dc` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:04.492333 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1b41dd-094e-4a9c-ad19-f1ac77bdb6dd None admin] RBAC: Authorizing `identity:get_project(project_id=c3ff9ff008b04f21a0d68698cf06e6dc)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:04.493823 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1b41dd-094e-4a9c-ad19-f1ac77bdb6dd None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:04.495885 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 146/293] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Aug 30 15:46:04 2022] GET /identity/v3/projects/c3ff9ff008b04f21a0d68698cf06e6dc => generated 302 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:46:04.501799 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fd50ed-d84c-4a60-be87-87e825c23ae5 None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:04.611373 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fd50ed-d84c-4a60-be87-87e825c23ae5 None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:04.632531 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:04.655149 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a24ba39-73d3-4a07-9c01-1c1c041f13a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:04.655807 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a24ba39-73d3-4a07-9c01-1c1c041f13a1 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:04.656373 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a24ba39-73d3-4a07-9c01-1c1c041f13a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:04.656666 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a24ba39-73d3-4a07-9c01-1c1c041f13a1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:04.657420 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a24ba39-73d3-4a07-9c01-1c1c041f13a1 None admin] RBAC: Adding query filter params (role.id=df8f9f844b1b439181d75a07c1a93418, scope.project.id=c3ff9ff008b04f21a0d68698cf06e6dc, user.id=470f03f3d02440ad9ffb32c7b79e1743) {{(pid=81512) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:46:04.657723 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a24ba39-73d3-4a07-9c01-1c1c041f13a1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:04.659152 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a24ba39-73d3-4a07-9c01-1c1c041f13a1 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:04.665125 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 148/294] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Aug 30 15:46:04 2022] GET /identity/v3/role_assignments?user.id=470f03f3d02440ad9ffb32c7b79e1743&scope.project.id=c3ff9ff008b04f21a0d68698cf06e6dc&role.id=df8f9f844b1b439181d75a07c1a93418 => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:46:04.771281 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570e79ac-94aa-4a64-9113-5768a84b7e01 None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:04.771796 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570e79ac-94aa-4a64-9113-5768a84b7e01 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:04.771939 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570e79ac-94aa-4a64-9113-5768a84b7e01 None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:04.773696 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 147/295] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 30 15:46:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 15:46:04.779031 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24116b0c-476f-4b13-9e0c-9a06358c0370 None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:04.779319 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24116b0c-476f-4b13-9e0c-9a06358c0370 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:04.779585 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24116b0c-476f-4b13-9e0c-9a06358c0370 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:04.806338 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-24116b0c-476f-4b13-9e0c-9a06358c0370 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:46:04.807569 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-24116b0c-476f-4b13-9e0c-9a06358c0370 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:46:04.891970 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 149/296] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 30 15:46:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 15:46:04.938662 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8389f4-23e3-4276-bafc-0fa37de06e9d None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:04.939000 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8389f4-23e3-4276-bafc-0fa37de06e9d None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:04.939192 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8389f4-23e3-4276-bafc-0fa37de06e9d None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:04.940914 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 148/297] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 30 15:46:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 15:46:04.946674 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e81c9d-0012-4925-9f01-d60af1db72ad None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:04.947009 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e81c9d-0012-4925-9f01-d60af1db72ad None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:04.947217 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e81c9d-0012-4925-9f01-d60af1db72ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:04.972071 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-75e81c9d-0012-4925-9f01-d60af1db72ad None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:46:04.973358 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-75e81c9d-0012-4925-9f01-d60af1db72ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:46:05.048517 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 150/298] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 30 15:46:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 15:46:05.203415 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8389f4-23e3-4276-bafc-0fa37de06e9d None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:05.319716 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8389f4-23e3-4276-bafc-0fa37de06e9d None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:05.323285 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e81c9d-0012-4925-9f01-d60af1db72ad None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:05.333029 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:05.371269 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a668b3-4e2a-406e-92c9-af0b7032a195 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:05.372277 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a668b3-4e2a-406e-92c9-af0b7032a195 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:05.372551 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a668b3-4e2a-406e-92c9-af0b7032a195 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:05.372815 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a668b3-4e2a-406e-92c9-af0b7032a195 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:05.373957 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a668b3-4e2a-406e-92c9-af0b7032a195 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:05.375950 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a668b3-4e2a-406e-92c9-af0b7032a195 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:05.378319 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 149/299] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 30 15:46:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:46:05.381672 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:05.384592 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a668b3-4e2a-406e-92c9-af0b7032a195 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:05.439861 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e81c9d-0012-4925-9f01-d60af1db72ad None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:05.473924 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149c93fb-4581-4f3d-b9e0-37281243ab22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:05.474556 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149c93fb-4581-4f3d-b9e0-37281243ab22 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:05.474838 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149c93fb-4581-4f3d-b9e0-37281243ab22 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:05.475100 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149c93fb-4581-4f3d-b9e0-37281243ab22 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:05.485103 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149c93fb-4581-4f3d-b9e0-37281243ab22 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 15:46:05.489941 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application [None req-149c93fb-4581-4f3d-b9e0-37281243ab22 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 30 15:46:05.489941 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:46:05.489941 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:46:05.489941 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:46:05.489941 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:46:05.489941 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:46:05.489941 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:46:05.489941 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:46:05.489941 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:46:05.489941 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:46:05.489941 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:46:05.489941 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:46:05.489941 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 15:46:05.489941 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 15:46:05.489941 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 15:46:05.489941 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 15:46:05.490827 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 15:46:05.490827 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 15:46:05.490827 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 15:46:05.490827 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 15:46:05.490827 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:46:05.490827 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:46:05.490827 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:46:05.490827 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:46:05.490827 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:46:05.490827 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:46:05.490827 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:46:05.490827 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application with Lock( Aug 30 15:46:05.490827 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:46:05.490827 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:46:05.490827 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:46:05.490827 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:46:05.491649 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:46:05.491649 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:46:05.491649 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:46:05.491649 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:46:05.491649 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 15:46:05.491649 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 15:46:05.491649 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 15:46:05.491649 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 15:46:05.491649 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 15:46:05.491649 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 15:46:05.491649 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 30 15:46:05.491649 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Aug 30 15:46:05.492253 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a668b3-4e2a-406e-92c9-af0b7032a195 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:05.492325 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 151/300] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Aug 30 15:46:05 2022] GET /identity/v3/roles/reader => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 15:46:05.493168 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce865c0e-c778-455f-8c9c-e4131f96a421 None None] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:05.493446 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce865c0e-c778-455f-8c9c-e4131f96a421 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:05.493719 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce865c0e-c778-455f-8c9c-e4131f96a421 None None] PATH_INFO: `/` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:05.495095 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 152/301] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 30 15:46:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 15:46:05.500118 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce865c0e-c778-455f-8c9c-e4131f96a421 None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:05.533615 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1397f54-297b-44a0-99b5-bba4a174c4bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:05.534226 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1397f54-297b-44a0-99b5-bba4a174c4bf None admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:05.534497 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1397f54-297b-44a0-99b5-bba4a174c4bf None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:05.534757 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1397f54-297b-44a0-99b5-bba4a174c4bf None admin] PATH_INFO: `/v3/groups` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:05.535493 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1397f54-297b-44a0-99b5-bba4a174c4bf None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:05.537252 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1397f54-297b-44a0-99b5-bba4a174c4bf None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:05.558948 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 150/302] 10.222.0.20 () {60 vars in 1134 bytes} [Tue Aug 30 15:46:05 2022] POST /identity/v3/groups => generated 229 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 15:46:05.560451 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b8f5b9-f40d-41bb-ad37-6398d0c15a5d None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:05.560735 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b8f5b9-f40d-41bb-ad37-6398d0c15a5d None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:05.561263 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b8f5b9-f40d-41bb-ad37-6398d0c15a5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:05.584146 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-51b8f5b9-f40d-41bb-ad37-6398d0c15a5d None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:46:05.585952 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-51b8f5b9-f40d-41bb-ad37-6398d0c15a5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:46:05.625519 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:05.655949 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce865c0e-c778-455f-8c9c-e4131f96a421 None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:05.661071 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:05.684307 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 151/303] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 30 15:46:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 15:46:05.703953 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a9cbfb-4b86-4c3b-b15f-78c5295ea091 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:05.704926 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a9cbfb-4b86-4c3b-b15f-78c5295ea091 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:05.705494 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a9cbfb-4b86-4c3b-b15f-78c5295ea091 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:05.705937 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a9cbfb-4b86-4c3b-b15f-78c5295ea091 None admin] PATH_INFO: `/v3/roles` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:05.706771 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a9cbfb-4b86-4c3b-b15f-78c5295ea091 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81512) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:46:05.707323 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a9cbfb-4b86-4c3b-b15f-78c5295ea091 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:05.709028 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a9cbfb-4b86-4c3b-b15f-78c5295ea091 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:05.718618 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 153/304] 10.222.0.20 () {56 vars in 1101 bytes} [Tue Aug 30 15:46:05 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:46:05.724473 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b8f5b9-f40d-41bb-ad37-6398d0c15a5d None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:05.830259 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b8f5b9-f40d-41bb-ad37-6398d0c15a5d None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:05.841802 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead638a8-d187-4a46-9be7-eb1a95131fd1 None None] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:05.842094 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead638a8-d187-4a46-9be7-eb1a95131fd1 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:05.842278 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead638a8-d187-4a46-9be7-eb1a95131fd1 None None] PATH_INFO: `/` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:05.843917 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 154/305] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 30 15:46:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 15:46:05.848752 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa4a6d0-b5b5-4482-84da-e6dcb6a959aa None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:05.849034 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa4a6d0-b5b5-4482-84da-e6dcb6a959aa None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:05.849336 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa4a6d0-b5b5-4482-84da-e6dcb6a959aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:05.870212 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-bfa4a6d0-b5b5-4482-84da-e6dcb6a959aa None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:46:05.870857 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9295078b-02d6-4baf-aa02-4c3a133152e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:05.871747 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-bfa4a6d0-b5b5-4482-84da-e6dcb6a959aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:46:05.872273 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9295078b-02d6-4baf-aa02-4c3a133152e8 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:05.872768 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9295078b-02d6-4baf-aa02-4c3a133152e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:05.873255 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9295078b-02d6-4baf-aa02-4c3a133152e8 None admin] PATH_INFO: `/v3/users/48da79d776614839bcc9aab9cf039c7b` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:05.887680 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9295078b-02d6-4baf-aa02-4c3a133152e8 None admin] RBAC: Authorizing `identity:get_user(user_id=48da79d776614839bcc9aab9cf039c7b)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:05.889537 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9295078b-02d6-4baf-aa02-4c3a133152e8 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:05.891791 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 152/306] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Aug 30 15:46:05 2022] GET /identity/v3/users/48da79d776614839bcc9aab9cf039c7b => generated 293 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:46:05.897736 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9295078b-02d6-4baf-aa02-4c3a133152e8 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:05.924529 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:05.958038 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 155/307] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 30 15:46:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 15:46:05.966183 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa4a6d0-b5b5-4482-84da-e6dcb6a959aa None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:06.013927 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9295078b-02d6-4baf-aa02-4c3a133152e8 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:06.050641 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc249356-7b97-4161-98f8-96c6a7f50600 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:06.051333 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc249356-7b97-4161-98f8-96c6a7f50600 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:06.051589 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc249356-7b97-4161-98f8-96c6a7f50600 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:06.051852 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc249356-7b97-4161-98f8-96c6a7f50600 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:06.052587 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc249356-7b97-4161-98f8-96c6a7f50600 None admin] RBAC: Adding query filter params (role.id=5d293e67c8e747e6910e95cf65e56633, user.id=48da79d776614839bcc9aab9cf039c7b, scope.system=all) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:46:06.052898 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc249356-7b97-4161-98f8-96c6a7f50600 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:06.054386 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc249356-7b97-4161-98f8-96c6a7f50600 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:06.064583 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 153/308] 10.222.0.20 () {56 vars in 1308 bytes} [Tue Aug 30 15:46:05 2022] GET /identity/v3/role_assignments?user.id=48da79d776614839bcc9aab9cf039c7b&scope.system=all&role.id=5d293e67c8e747e6910e95cf65e56633 => generated 226 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 15:46:06.091147 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:06.098687 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa4a6d0-b5b5-4482-84da-e6dcb6a959aa None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:06.137030 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e93965-6719-4512-8213-75d7b93e6e1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:06.138160 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e93965-6719-4512-8213-75d7b93e6e1d None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:06.138631 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e93965-6719-4512-8213-75d7b93e6e1d None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:06.139090 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e93965-6719-4512-8213-75d7b93e6e1d None admin] PATH_INFO: `/v3/roles/member` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:06.151496 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e93965-6719-4512-8213-75d7b93e6e1d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 15:46:06.157020 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application [None req-a9e93965-6719-4512-8213-75d7b93e6e1d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 30 15:46:06.157020 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:46:06.157020 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:46:06.157020 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:46:06.157020 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:46:06.157020 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:46:06.157020 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:46:06.157020 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:46:06.157020 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:46:06.157020 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:46:06.157020 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:46:06.157020 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:46:06.157020 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 15:46:06.157020 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 15:46:06.157020 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 15:46:06.157020 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 15:46:06.158218 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 15:46:06.158218 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 15:46:06.158218 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 15:46:06.158218 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 15:46:06.158218 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:46:06.158218 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:46:06.158218 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:46:06.158218 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:46:06.158218 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:46:06.158218 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:46:06.158218 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:46:06.158218 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application with Lock( Aug 30 15:46:06.158218 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:46:06.158218 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:46:06.158218 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:46:06.158218 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:46:06.159022 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:46:06.159022 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:46:06.159022 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:46:06.159022 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:46:06.159022 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 15:46:06.159022 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 15:46:06.159022 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 15:46:06.159022 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 15:46:06.159022 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 15:46:06.159022 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 15:46:06.159022 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 30 15:46:06.159022 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Aug 30 15:46:06.165311 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 156/309] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Aug 30 15:46:05 2022] GET /identity/v3/roles/member => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 15:46:06.171745 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc249356-7b97-4161-98f8-96c6a7f50600 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:06.231428 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e93965-6719-4512-8213-75d7b93e6e1d None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:06.276044 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:06.291584 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc249356-7b97-4161-98f8-96c6a7f50600 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:06.335708 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b99cacf-6110-4982-82d3-8c2d2f8255e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:06.337033 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b99cacf-6110-4982-82d3-8c2d2f8255e8 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:06.337538 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b99cacf-6110-4982-82d3-8c2d2f8255e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:06.338003 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b99cacf-6110-4982-82d3-8c2d2f8255e8 None admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:06.339054 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b99cacf-6110-4982-82d3-8c2d2f8255e8 None admin] RBAC: Adding query filter params (name=member) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:46:06.339543 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b99cacf-6110-4982-82d3-8c2d2f8255e8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:06.341767 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b99cacf-6110-4982-82d3-8c2d2f8255e8 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:06.357208 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 154/310] 10.222.0.20 () {56 vars in 1101 bytes} [Tue Aug 30 15:46:06 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:46:06.359364 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b99cacf-6110-4982-82d3-8c2d2f8255e8 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:06.364922 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e93965-6719-4512-8213-75d7b93e6e1d None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:06.409549 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5f93b2-3c96-4dae-ae27-d4d558687479 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:06.410728 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5f93b2-3c96-4dae-ae27-d4d558687479 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:06.411018 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5f93b2-3c96-4dae-ae27-d4d558687479 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:06.411287 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5f93b2-3c96-4dae-ae27-d4d558687479 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:06.424805 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5f93b2-3c96-4dae-ae27-d4d558687479 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 15:46:06.431807 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application [None req-8b5f93b2-3c96-4dae-ae27-d4d558687479 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 30 15:46:06.431807 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:46:06.431807 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:46:06.431807 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:46:06.431807 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:46:06.431807 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:46:06.431807 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:46:06.431807 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:46:06.431807 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:46:06.431807 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:46:06.431807 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:46:06.431807 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:46:06.431807 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 15:46:06.431807 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 15:46:06.431807 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 15:46:06.431807 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 15:46:06.432693 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 15:46:06.432693 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 15:46:06.432693 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 15:46:06.432693 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 15:46:06.432693 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:46:06.432693 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:46:06.432693 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:46:06.432693 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:46:06.432693 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:46:06.432693 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:46:06.432693 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:46:06.432693 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application with Lock( Aug 30 15:46:06.432693 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:46:06.432693 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:46:06.432693 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:46:06.432693 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:46:06.433549 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:46:06.433549 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:46:06.433549 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:46:06.433549 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:46:06.433549 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 15:46:06.433549 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 15:46:06.433549 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 15:46:06.433549 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 15:46:06.433549 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 15:46:06.433549 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 15:46:06.433549 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 30 15:46:06.433549 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Aug 30 15:46:06.437837 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 157/311] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Aug 30 15:46:06 2022] GET /identity/v3/roles/reader => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 15:46:06.438656 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5f93b2-3c96-4dae-ae27-d4d558687479 None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:06.458140 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:06.486995 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b99cacf-6110-4982-82d3-8c2d2f8255e8 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:06.523645 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aed1244-3080-44a2-bf21-f9b9c3a34b73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:06.524640 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aed1244-3080-44a2-bf21-f9b9c3a34b73 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:06.525110 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aed1244-3080-44a2-bf21-f9b9c3a34b73 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:06.525620 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aed1244-3080-44a2-bf21-f9b9c3a34b73 None admin] PATH_INFO: `/v3/users/1c5d0a4e865d4bc38d4bfa8256548d3c` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:06.527176 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aed1244-3080-44a2-bf21-f9b9c3a34b73 None admin] RBAC: Authorizing `identity:get_user(user_id=1c5d0a4e865d4bc38d4bfa8256548d3c)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:06.528813 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aed1244-3080-44a2-bf21-f9b9c3a34b73 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:06.530874 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 155/312] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Aug 30 15:46:06 2022] GET /identity/v3/users/1c5d0a4e865d4bc38d4bfa8256548d3c => generated 293 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:46:06.537445 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aed1244-3080-44a2-bf21-f9b9c3a34b73 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:06.545468 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5f93b2-3c96-4dae-ae27-d4d558687479 None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:06.557902 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:06.582274 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272c621a-0dec-4eab-b173-cf7396ee81ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:06.582882 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272c621a-0dec-4eab-b173-cf7396ee81ca None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:06.583156 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272c621a-0dec-4eab-b173-cf7396ee81ca None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:06.583421 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272c621a-0dec-4eab-b173-cf7396ee81ca None admin] PATH_INFO: `/v3/roles` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:06.584082 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272c621a-0dec-4eab-b173-cf7396ee81ca None admin] RBAC: Adding query filter params (name=reader) {{(pid=81512) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:46:06.584380 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272c621a-0dec-4eab-b173-cf7396ee81ca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:06.585789 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272c621a-0dec-4eab-b173-cf7396ee81ca None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:06.599040 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 158/313] 10.222.0.20 () {56 vars in 1101 bytes} [Tue Aug 30 15:46:06 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 15:46:06.600009 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272c621a-0dec-4eab-b173-cf7396ee81ca None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:06.667708 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aed1244-3080-44a2-bf21-f9b9c3a34b73 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:06.695589 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:06.716108 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7787c884-2a0b-427d-9fe2-3dde18f446a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:06.717274 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7787c884-2a0b-427d-9fe2-3dde18f446a7 None admin] REQUEST_METHOD: `PUT` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:06.717652 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7787c884-2a0b-427d-9fe2-3dde18f446a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:06.718021 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7787c884-2a0b-427d-9fe2-3dde18f446a7 None admin] PATH_INFO: `/v3/system/users/1c5d0a4e865d4bc38d4bfa8256548d3c/roles/df8f9f844b1b439181d75a07c1a93418` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:06.720565 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7787c884-2a0b-427d-9fe2-3dde18f446a7 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=1c5d0a4e865d4bc38d4bfa8256548d3c, role_id=df8f9f844b1b439181d75a07c1a93418)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:06.721685 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272c621a-0dec-4eab-b173-cf7396ee81ca None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:06.722317 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7787c884-2a0b-427d-9fe2-3dde18f446a7 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:06.744181 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 156/314] 10.222.0.20 () {58 vars in 1334 bytes} [Tue Aug 30 15:46:06 2022] PUT /identity/v3/system/users/1c5d0a4e865d4bc38d4bfa8256548d3c/roles/df8f9f844b1b439181d75a07c1a93418 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:46:06.769864 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afad599-711d-4e89-902a-3c6f0f36ac26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:06.770779 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afad599-711d-4e89-902a-3c6f0f36ac26 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:06.771238 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afad599-711d-4e89-902a-3c6f0f36ac26 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:06.771743 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afad599-711d-4e89-902a-3c6f0f36ac26 None admin] PATH_INFO: `/v3/users/25ef42035fee4b54b04e59d194820e7b` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:06.773258 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afad599-711d-4e89-902a-3c6f0f36ac26 None admin] RBAC: Authorizing `identity:get_user(user_id=25ef42035fee4b54b04e59d194820e7b)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:06.774885 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afad599-711d-4e89-902a-3c6f0f36ac26 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:06.777184 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 159/315] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Aug 30 15:46:06 2022] GET /identity/v3/users/25ef42035fee4b54b04e59d194820e7b => generated 297 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 15:46:06.783426 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7787c884-2a0b-427d-9fe2-3dde18f446a7 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:06.883428 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:06.903803 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7787c884-2a0b-427d-9fe2-3dde18f446a7 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:06.941513 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69450f63-e80f-421b-9de0-632ebc388a2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:06.942130 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69450f63-e80f-421b-9de0-632ebc388a2e None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:06.942609 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69450f63-e80f-421b-9de0-632ebc388a2e None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:06.942880 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69450f63-e80f-421b-9de0-632ebc388a2e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:07.015492 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69450f63-e80f-421b-9de0-632ebc388a2e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:07.016988 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69450f63-e80f-421b-9de0-632ebc388a2e None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:07.103908 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:07.124895 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 157/316] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Aug 30 15:46:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 15:46:07.131082 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afad599-711d-4e89-902a-3c6f0f36ac26 None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:07.243156 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afad599-711d-4e89-902a-3c6f0f36ac26 None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:07.280721 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776cc7e8-5dd8-4ee2-a42c-e5b39624b298 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:07.281396 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776cc7e8-5dd8-4ee2-a42c-e5b39624b298 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:07.281671 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776cc7e8-5dd8-4ee2-a42c-e5b39624b298 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:07.281940 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776cc7e8-5dd8-4ee2-a42c-e5b39624b298 None admin] PATH_INFO: `/v3/projects/c3ff9ff008b04f21a0d68698cf06e6dc` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:07.283064 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776cc7e8-5dd8-4ee2-a42c-e5b39624b298 None admin] RBAC: Authorizing `identity:get_project(project_id=c3ff9ff008b04f21a0d68698cf06e6dc)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:07.284502 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776cc7e8-5dd8-4ee2-a42c-e5b39624b298 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:07.286506 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 160/317] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Aug 30 15:46:07 2022] GET /identity/v3/projects/c3ff9ff008b04f21a0d68698cf06e6dc => generated 302 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 15:46:07.294020 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69450f63-e80f-421b-9de0-632ebc388a2e None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:07.395936 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69450f63-e80f-421b-9de0-632ebc388a2e None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:07.431044 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a0dc59-571f-4b03-801c-b1312713b288 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:07.432047 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a0dc59-571f-4b03-801c-b1312713b288 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:07.432519 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a0dc59-571f-4b03-801c-b1312713b288 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:07.432982 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a0dc59-571f-4b03-801c-b1312713b288 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:07.433935 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a0dc59-571f-4b03-801c-b1312713b288 None admin] RBAC: Adding query filter params (role.id=5d293e67c8e747e6910e95cf65e56633, scope.project.id=c3ff9ff008b04f21a0d68698cf06e6dc, user.id=25ef42035fee4b54b04e59d194820e7b) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:46:07.434450 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a0dc59-571f-4b03-801c-b1312713b288 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:07.436089 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a0dc59-571f-4b03-801c-b1312713b288 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:07.442999 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 158/318] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Aug 30 15:46:07 2022] GET /identity/v3/role_assignments?user.id=25ef42035fee4b54b04e59d194820e7b&scope.project.id=c3ff9ff008b04f21a0d68698cf06e6dc&role.id=5d293e67c8e747e6910e95cf65e56633 => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:46:07.556592 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3141b988-3bba-4c68-a46c-45b68410b07c None None] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:07.556939 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3141b988-3bba-4c68-a46c-45b68410b07c None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:07.557056 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3141b988-3bba-4c68-a46c-45b68410b07c None None] PATH_INFO: `/` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:07.558780 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 161/319] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 30 15:46:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 15:46:07.564473 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0f234b-2e7d-4aac-b124-e40c16fdafa5 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:07.564759 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0f234b-2e7d-4aac-b124-e40c16fdafa5 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:07.565027 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0f234b-2e7d-4aac-b124-e40c16fdafa5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:07.583936 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-8a0f234b-2e7d-4aac-b124-e40c16fdafa5 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:46:07.585441 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-8a0f234b-2e7d-4aac-b124-e40c16fdafa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:46:07.634243 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:07.658499 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 159/320] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 30 15:46:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 15:46:07.923040 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3141b988-3bba-4c68-a46c-45b68410b07c None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:07.941426 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:08.035021 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3141b988-3bba-4c68-a46c-45b68410b07c None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:08.039884 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ff141d-3188-4011-8b43-d5fbb172657e None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:08.040333 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ff141d-3188-4011-8b43-d5fbb172657e None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:08.040406 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ff141d-3188-4011-8b43-d5fbb172657e None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:08.041941 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 160/321] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 30 15:46:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 15:46:08.046604 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d666e0-6c11-4797-a3c8-6f836cd24b75 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:08.046896 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d666e0-6c11-4797-a3c8-6f836cd24b75 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:08.047154 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d666e0-6c11-4797-a3c8-6f836cd24b75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:08.064868 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-c1d666e0-6c11-4797-a3c8-6f836cd24b75 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:46:08.066100 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-c1d666e0-6c11-4797-a3c8-6f836cd24b75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:46:08.069157 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0a0bd7-a4cc-4d05-9147-89a7f8586e8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:08.069837 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0a0bd7-a4cc-4d05-9147-89a7f8586e8b None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:08.070114 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0a0bd7-a4cc-4d05-9147-89a7f8586e8b None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:08.070381 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0a0bd7-a4cc-4d05-9147-89a7f8586e8b None admin] PATH_INFO: `/v3/domains/default` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:08.071461 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0a0bd7-a4cc-4d05-9147-89a7f8586e8b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:08.072899 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0a0bd7-a4cc-4d05-9147-89a7f8586e8b None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:08.074655 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 162/322] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 30 15:46:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:46:08.080175 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0a0bd7-a4cc-4d05-9147-89a7f8586e8b None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:08.142468 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 161/323] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 30 15:46:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 15:46:08.166438 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:08.189386 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0a0bd7-a4cc-4d05-9147-89a7f8586e8b None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:08.222809 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8514d211-fea7-4c27-8c8d-85ba77966c93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:08.223415 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8514d211-fea7-4c27-8c8d-85ba77966c93 None admin] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:08.223691 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8514d211-fea7-4c27-8c8d-85ba77966c93 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:08.223956 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8514d211-fea7-4c27-8c8d-85ba77966c93 None admin] PATH_INFO: `/v3/groups` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:08.224608 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8514d211-fea7-4c27-8c8d-85ba77966c93 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:08.226083 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8514d211-fea7-4c27-8c8d-85ba77966c93 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:08.245190 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 163/324] 10.222.0.20 () {60 vars in 1134 bytes} [Tue Aug 30 15:46:08 2022] POST /identity/v3/groups => generated 226 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 15:46:08.408960 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d666e0-6c11-4797-a3c8-6f836cd24b75 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:08.516595 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d666e0-6c11-4797-a3c8-6f836cd24b75 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:08.525002 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:08.554445 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cf5485-5438-4334-a3a1-1a2a94cbae89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:08.555101 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cf5485-5438-4334-a3a1-1a2a94cbae89 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:08.555378 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cf5485-5438-4334-a3a1-1a2a94cbae89 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:08.555646 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cf5485-5438-4334-a3a1-1a2a94cbae89 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:08.566457 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37cf5485-5438-4334-a3a1-1a2a94cbae89 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 15:46:08.571381 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-37cf5485-5438-4334-a3a1-1a2a94cbae89 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 30 15:46:08.571381 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:46:08.571381 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:46:08.571381 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:46:08.571381 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:46:08.571381 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:46:08.571381 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:46:08.571381 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:46:08.571381 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:46:08.571381 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:46:08.571381 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:46:08.571381 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:46:08.571381 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 15:46:08.571381 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 15:46:08.571381 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 15:46:08.571381 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 15:46:08.572265 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 15:46:08.572265 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 15:46:08.572265 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 15:46:08.572265 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 15:46:08.572265 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:46:08.572265 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:46:08.572265 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:46:08.572265 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:46:08.572265 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:46:08.572265 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:46:08.572265 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:46:08.572265 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Aug 30 15:46:08.572265 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:46:08.572265 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:46:08.572265 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:46:08.572265 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:46:08.573059 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:46:08.573059 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:46:08.573059 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:46:08.573059 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:46:08.573059 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 15:46:08.573059 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 15:46:08.573059 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 15:46:08.573059 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 15:46:08.573059 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 15:46:08.573059 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 15:46:08.573059 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 30 15:46:08.573059 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Aug 30 15:46:08.573059 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 162/325] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Aug 30 15:46:08 2022] GET /identity/v3/roles/reader => generated 84 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 15:46:08.578790 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8514d211-fea7-4c27-8c8d-85ba77966c93 None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:08.697442 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8514d211-fea7-4c27-8c8d-85ba77966c93 None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:08.733310 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6455445b-eae6-4bf7-9035-0ec8359412a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:08.733923 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6455445b-eae6-4bf7-9035-0ec8359412a2 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:08.734211 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6455445b-eae6-4bf7-9035-0ec8359412a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:08.734483 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6455445b-eae6-4bf7-9035-0ec8359412a2 None admin] PATH_INFO: `/v3/roles` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:08.735187 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6455445b-eae6-4bf7-9035-0ec8359412a2 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81512) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:46:08.735501 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6455445b-eae6-4bf7-9035-0ec8359412a2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:08.736900 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6455445b-eae6-4bf7-9035-0ec8359412a2 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:08.746673 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 164/326] 10.222.0.20 () {56 vars in 1101 bytes} [Tue Aug 30 15:46:08 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 15:46:08.752234 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cf5485-5438-4334-a3a1-1a2a94cbae89 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:08.773179 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245cac0d-724b-40b7-af30-f53a304104b2 None None] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:08.773457 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245cac0d-724b-40b7-af30-f53a304104b2 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:08.773693 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245cac0d-724b-40b7-af30-f53a304104b2 None None] PATH_INFO: `/` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:08.775297 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 165/327] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 30 15:46:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 15:46:08.780463 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf38d0f-d3ff-4a0e-89e9-8d653ecbf079 None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:08.780622 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf38d0f-d3ff-4a0e-89e9-8d653ecbf079 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:08.780933 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf38d0f-d3ff-4a0e-89e9-8d653ecbf079 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:08.789426 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:08.802419 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-4bf38d0f-d3ff-4a0e-89e9-8d653ecbf079 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:46:08.803760 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-4bf38d0f-d3ff-4a0e-89e9-8d653ecbf079 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:46:08.881635 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cf5485-5438-4334-a3a1-1a2a94cbae89 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:08.886365 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 166/328] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 30 15:46:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 15:46:08.930852 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb606e7-b301-4fe8-9407-ab928a5abd5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:08.931862 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb606e7-b301-4fe8-9407-ab928a5abd5e None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:08.932331 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb606e7-b301-4fe8-9407-ab928a5abd5e None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:08.932807 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb606e7-b301-4fe8-9407-ab928a5abd5e None admin] PATH_INFO: `/v3/users/48da79d776614839bcc9aab9cf039c7b` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:08.934231 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb606e7-b301-4fe8-9407-ab928a5abd5e None admin] RBAC: Authorizing `identity:get_user(user_id=48da79d776614839bcc9aab9cf039c7b)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:08.935888 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb606e7-b301-4fe8-9407-ab928a5abd5e None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:08.938167 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 163/329] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Aug 30 15:46:08 2022] GET /identity/v3/users/48da79d776614839bcc9aab9cf039c7b => generated 293 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:46:08.944007 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf38d0f-d3ff-4a0e-89e9-8d653ecbf079 None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:09.056145 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf38d0f-d3ff-4a0e-89e9-8d653ecbf079 None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:09.092038 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8d6fdc-bdb9-43c2-9b72-ba2124a72e8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:09.093039 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8d6fdc-bdb9-43c2-9b72-ba2124a72e8d None admin] REQUEST_METHOD: `PUT` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:09.093586 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8d6fdc-bdb9-43c2-9b72-ba2124a72e8d None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:09.094106 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8d6fdc-bdb9-43c2-9b72-ba2124a72e8d None admin] PATH_INFO: `/v3/system/users/48da79d776614839bcc9aab9cf039c7b/roles/5d293e67c8e747e6910e95cf65e56633` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:09.095843 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8d6fdc-bdb9-43c2-9b72-ba2124a72e8d None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=48da79d776614839bcc9aab9cf039c7b, role_id=5d293e67c8e747e6910e95cf65e56633)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:09.097511 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8d6fdc-bdb9-43c2-9b72-ba2124a72e8d None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:09.101532 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:09.115046 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 167/330] 10.222.0.20 () {58 vars in 1334 bytes} [Tue Aug 30 15:46:08 2022] PUT /identity/v3/system/users/48da79d776614839bcc9aab9cf039c7b/roles/5d293e67c8e747e6910e95cf65e56633 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:46:09.290207 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb606e7-b301-4fe8-9407-ab928a5abd5e None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:09.334335 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:09.412493 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb606e7-b301-4fe8-9407-ab928a5abd5e None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:09.450653 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ec7c3a-512d-45ab-a9de-af3ef5e28617 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:09.451675 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ec7c3a-512d-45ab-a9de-af3ef5e28617 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:09.452142 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ec7c3a-512d-45ab-a9de-af3ef5e28617 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:09.452656 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ec7c3a-512d-45ab-a9de-af3ef5e28617 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:09.469761 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ec7c3a-512d-45ab-a9de-af3ef5e28617 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 15:46:09.475043 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-89ec7c3a-512d-45ab-a9de-af3ef5e28617 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 30 15:46:09.475043 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:46:09.475043 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:46:09.475043 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:46:09.475043 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:46:09.475043 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:46:09.475043 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:46:09.475043 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:46:09.475043 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:46:09.475043 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:46:09.475043 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:46:09.475043 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:46:09.475043 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 15:46:09.475043 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 15:46:09.475043 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 15:46:09.475043 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 15:46:09.475956 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 15:46:09.475956 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 15:46:09.475956 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 15:46:09.475956 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 15:46:09.475956 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:46:09.475956 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:46:09.475956 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:46:09.475956 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:46:09.475956 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:46:09.475956 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:46:09.475956 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:46:09.475956 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Aug 30 15:46:09.475956 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:46:09.475956 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:46:09.475956 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:46:09.475956 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:46:09.476759 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:46:09.476759 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:46:09.476759 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:46:09.476759 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:46:09.476759 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 15:46:09.476759 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 15:46:09.476759 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 15:46:09.476759 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 15:46:09.476759 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 15:46:09.476759 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 15:46:09.476759 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 30 15:46:09.476759 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Aug 30 15:46:09.479061 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 164/331] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Aug 30 15:46:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 15:46:09.485413 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8d6fdc-bdb9-43c2-9b72-ba2124a72e8d None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:09.616340 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8d6fdc-bdb9-43c2-9b72-ba2124a72e8d None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:09.659556 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29bcc5d-652a-4121-aa3a-96401dab1864 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:09.660176 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29bcc5d-652a-4121-aa3a-96401dab1864 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:09.660454 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29bcc5d-652a-4121-aa3a-96401dab1864 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:09.660722 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29bcc5d-652a-4121-aa3a-96401dab1864 None admin] PATH_INFO: `/v3/roles` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:09.661427 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29bcc5d-652a-4121-aa3a-96401dab1864 None admin] RBAC: Adding query filter params (name=member) {{(pid=81512) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:46:09.661726 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29bcc5d-652a-4121-aa3a-96401dab1864 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:09.663114 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29bcc5d-652a-4121-aa3a-96401dab1864 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:09.671898 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 168/332] 10.222.0.20 () {56 vars in 1101 bytes} [Tue Aug 30 15:46:09 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:46:09.677411 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ec7c3a-512d-45ab-a9de-af3ef5e28617 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:09.789645 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:09.796489 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ec7c3a-512d-45ab-a9de-af3ef5e28617 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:09.831583 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d84cae-83fb-477a-826c-5d8f88e0ddd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:09.832279 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d84cae-83fb-477a-826c-5d8f88e0ddd1 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:09.832558 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d84cae-83fb-477a-826c-5d8f88e0ddd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:09.832829 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d84cae-83fb-477a-826c-5d8f88e0ddd1 None admin] PATH_INFO: `/v3/users/1c5d0a4e865d4bc38d4bfa8256548d3c` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:09.837805 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d84cae-83fb-477a-826c-5d8f88e0ddd1 None admin] RBAC: Authorizing `identity:get_user(user_id=1c5d0a4e865d4bc38d4bfa8256548d3c)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:09.839282 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d84cae-83fb-477a-826c-5d8f88e0ddd1 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:09.841321 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 165/333] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Aug 30 15:46:09 2022] GET /identity/v3/users/1c5d0a4e865d4bc38d4bfa8256548d3c => generated 293 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:46:09.847090 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29bcc5d-652a-4121-aa3a-96401dab1864 None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:09.922880 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:09.969148 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29bcc5d-652a-4121-aa3a-96401dab1864 None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:10.004168 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfb845a-da78-4e9d-a99a-49f801213c0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:10.004779 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfb845a-da78-4e9d-a99a-49f801213c0a None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:10.005119 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfb845a-da78-4e9d-a99a-49f801213c0a None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:10.005418 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfb845a-da78-4e9d-a99a-49f801213c0a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:10.006198 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdfb845a-da78-4e9d-a99a-49f801213c0a None admin] RBAC: Adding query filter params (role.id=df8f9f844b1b439181d75a07c1a93418, user.id=1c5d0a4e865d4bc38d4bfa8256548d3c, scope.system=all) {{(pid=81512) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:46:10.006558 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdfb845a-da78-4e9d-a99a-49f801213c0a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:10.008013 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdfb845a-da78-4e9d-a99a-49f801213c0a None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:10.022994 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 169/334] 10.222.0.20 () {56 vars in 1308 bytes} [Tue Aug 30 15:46:09 2022] GET /identity/v3/role_assignments?user.id=1c5d0a4e865d4bc38d4bfa8256548d3c&scope.system=all&role.id=df8f9f844b1b439181d75a07c1a93418 => generated 510 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:46:10.716828 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0081d8f-53b6-4eb9-8d40-402c78725b82 None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:10.717409 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0081d8f-53b6-4eb9-8d40-402c78725b82 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:10.717409 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0081d8f-53b6-4eb9-8d40-402c78725b82 None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:10.719194 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 166/335] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 30 15:46:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 15:46:10.725978 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910c2dfd-888b-4cd6-85a4-9a849dcea398 None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:10.726255 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910c2dfd-888b-4cd6-85a4-9a849dcea398 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:10.726521 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910c2dfd-888b-4cd6-85a4-9a849dcea398 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:10.758315 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-910c2dfd-888b-4cd6-85a4-9a849dcea398 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:46:10.759989 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-910c2dfd-888b-4cd6-85a4-9a849dcea398 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:46:10.842912 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 170/336] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 30 15:46:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 15:46:11.163892 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9551d009-8f76-4f91-a178-a03274f21ef4 None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:11.164168 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9551d009-8f76-4f91-a178-a03274f21ef4 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:11.164591 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9551d009-8f76-4f91-a178-a03274f21ef4 None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:11.166379 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 167/337] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 30 15:46:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 15:46:11.171469 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ea2163-8256-422a-bf98-4d79a4c312ba None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:11.171930 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ea2163-8256-422a-bf98-4d79a4c312ba None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:11.172006 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ea2163-8256-422a-bf98-4d79a4c312ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:11.191128 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-59ea2163-8256-422a-bf98-4d79a4c312ba None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:46:11.192340 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-59ea2163-8256-422a-bf98-4d79a4c312ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:46:11.254284 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:11.285908 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 171/338] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 30 15:46:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 15:46:11.339950 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9551d009-8f76-4f91-a178-a03274f21ef4 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:11.463655 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9551d009-8f76-4f91-a178-a03274f21ef4 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:11.506738 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8d3c59-9832-4374-9667-f22bc2249cba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:11.507743 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8d3c59-9832-4374-9667-f22bc2249cba None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:11.508240 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8d3c59-9832-4374-9667-f22bc2249cba None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:11.508691 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8d3c59-9832-4374-9667-f22bc2249cba None admin] PATH_INFO: `/v3/roles/member` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:11.525311 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8d3c59-9832-4374-9667-f22bc2249cba None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 15:46:11.530662 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-1b8d3c59-9832-4374-9667-f22bc2249cba None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 30 15:46:11.530662 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:46:11.530662 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:46:11.530662 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:46:11.530662 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:46:11.530662 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:46:11.530662 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:46:11.530662 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:46:11.530662 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:46:11.530662 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:46:11.530662 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:46:11.530662 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:46:11.530662 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 15:46:11.530662 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 15:46:11.530662 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 15:46:11.530662 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 15:46:11.531575 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 15:46:11.531575 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 15:46:11.531575 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 15:46:11.531575 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 15:46:11.531575 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:46:11.531575 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:46:11.531575 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:46:11.531575 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:46:11.531575 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:46:11.531575 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:46:11.531575 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:46:11.531575 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Aug 30 15:46:11.531575 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:46:11.531575 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:46:11.531575 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:46:11.531575 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:46:11.532389 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:46:11.532389 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:46:11.532389 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:46:11.532389 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:46:11.532389 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 15:46:11.532389 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 15:46:11.532389 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 15:46:11.532389 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 15:46:11.532389 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 15:46:11.532389 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 15:46:11.532389 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 30 15:46:11.532389 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Aug 30 15:46:11.534985 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 168/339] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Aug 30 15:46:11 2022] GET /identity/v3/roles/member => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 15:46:11.542802 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ea2163-8256-422a-bf98-4d79a4c312ba None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:11.688564 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ea2163-8256-422a-bf98-4d79a4c312ba None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:11.728516 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b947f76-bf52-4762-9f75-70e54dff70d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:11.729053 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b947f76-bf52-4762-9f75-70e54dff70d3 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:11.729385 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b947f76-bf52-4762-9f75-70e54dff70d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:11.729659 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b947f76-bf52-4762-9f75-70e54dff70d3 None admin] PATH_INFO: `/v3/roles` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:11.730340 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b947f76-bf52-4762-9f75-70e54dff70d3 None admin] RBAC: Adding query filter params (name=member) {{(pid=81512) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:46:11.730644 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b947f76-bf52-4762-9f75-70e54dff70d3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:11.732044 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b947f76-bf52-4762-9f75-70e54dff70d3 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:11.741063 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 172/340] 10.222.0.20 () {56 vars in 1101 bytes} [Tue Aug 30 15:46:11 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:46:11.747404 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8d3c59-9832-4374-9667-f22bc2249cba None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:11.755509 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfc8a9b-2632-457a-95cb-3fe6ca59ccc4 None None] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:11.756065 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfc8a9b-2632-457a-95cb-3fe6ca59ccc4 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:11.756581 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfc8a9b-2632-457a-95cb-3fe6ca59ccc4 None None] PATH_INFO: `/` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:11.758260 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 173/341] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 30 15:46:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 15:46:11.764096 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254fb10a-7fda-45d3-9a9e-f1c3fd78afca None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:11.764625 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254fb10a-7fda-45d3-9a9e-f1c3fd78afca None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:11.765169 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254fb10a-7fda-45d3-9a9e-f1c3fd78afca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:11.785073 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-254fb10a-7fda-45d3-9a9e-f1c3fd78afca None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:46:11.787241 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-254fb10a-7fda-45d3-9a9e-f1c3fd78afca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:46:11.797908 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:11.835607 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:11.873301 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 174/342] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 30 15:46:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 15:46:11.874518 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254fb10a-7fda-45d3-9a9e-f1c3fd78afca None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:11.996960 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8d3c59-9832-4374-9667-f22bc2249cba None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:12.038550 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254fb10a-7fda-45d3-9a9e-f1c3fd78afca None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:12.077990 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67198833-5464-481b-8038-36c75ffdcc7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:12.078624 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67198833-5464-481b-8038-36c75ffdcc7e None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:12.078956 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67198833-5464-481b-8038-36c75ffdcc7e None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:12.079227 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67198833-5464-481b-8038-36c75ffdcc7e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:12.105001 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d986809-7785-4ff6-b611-44e8bed1cc17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:12.105916 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d986809-7785-4ff6-b611-44e8bed1cc17 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:12.106333 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d986809-7785-4ff6-b611-44e8bed1cc17 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:12.106735 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d986809-7785-4ff6-b611-44e8bed1cc17 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:12.133867 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d986809-7785-4ff6-b611-44e8bed1cc17 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 15:46:12.138748 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application [None req-0d986809-7785-4ff6-b611-44e8bed1cc17 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 15:46:12.138748 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:46:12.138748 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:46:12.138748 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:46:12.138748 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:46:12.138748 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:46:12.138748 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:46:12.138748 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:46:12.138748 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:46:12.138748 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:46:12.138748 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:46:12.138748 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:46:12.138748 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 15:46:12.138748 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 15:46:12.138748 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 15:46:12.138748 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 15:46:12.139618 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 15:46:12.139618 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 15:46:12.139618 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 15:46:12.139618 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 15:46:12.139618 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:46:12.139618 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:46:12.139618 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:46:12.139618 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:46:12.139618 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:46:12.139618 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:46:12.139618 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:46:12.139618 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application with Lock( Aug 30 15:46:12.139618 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:46:12.139618 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:46:12.139618 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:46:12.139618 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:46:12.140462 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:46:12.140462 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:46:12.140462 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:46:12.140462 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:46:12.140462 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 15:46:12.140462 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 15:46:12.140462 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 15:46:12.140462 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 15:46:12.140462 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 15:46:12.140462 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 15:46:12.140462 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 15:46:12.140462 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Aug 30 15:46:12.140462 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 175/343] 10.222.0.20 () {56 vars in 1114 bytes} [Tue Aug 30 15:46:11 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 15:46:12.148623 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d986809-7785-4ff6-b611-44e8bed1cc17 None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:12.218290 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:12.233708 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67198833-5464-481b-8038-36c75ffdcc7e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:12.235219 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67198833-5464-481b-8038-36c75ffdcc7e None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:12.269676 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:12.346900 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d986809-7785-4ff6-b611-44e8bed1cc17 None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:12.395913 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 169/344] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Aug 30 15:46:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 648 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 15:46:12.396906 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67198833-5464-481b-8038-36c75ffdcc7e None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:12.397396 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539b946b-d853-440a-8fac-e1cf503bd45d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:12.398016 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539b946b-d853-440a-8fac-e1cf503bd45d None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:12.398292 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539b946b-d853-440a-8fac-e1cf503bd45d None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:12.398561 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539b946b-d853-440a-8fac-e1cf503bd45d None admin] PATH_INFO: `/v3/roles` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:12.399357 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539b946b-d853-440a-8fac-e1cf503bd45d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81512) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:46:12.399661 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539b946b-d853-440a-8fac-e1cf503bd45d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:12.401179 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539b946b-d853-440a-8fac-e1cf503bd45d None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:12.417177 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 176/345] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Aug 30 15:46:12 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 15:46:12.418138 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539b946b-d853-440a-8fac-e1cf503bd45d None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:12.540159 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67198833-5464-481b-8038-36c75ffdcc7e None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:12.548559 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:12.563370 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539b946b-d853-440a-8fac-e1cf503bd45d None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:12.571007 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:12.586898 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb9f346-7aad-4b94-8dfd-41463fa2f793 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:12.587803 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb9f346-7aad-4b94-8dfd-41463fa2f793 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:12.588079 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb9f346-7aad-4b94-8dfd-41463fa2f793 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:12.588347 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb9f346-7aad-4b94-8dfd-41463fa2f793 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:12.600254 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aad9b50-997c-4776-a3c7-84bc8361f7e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:12.600887 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aad9b50-997c-4776-a3c7-84bc8361f7e0 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:12.601178 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aad9b50-997c-4776-a3c7-84bc8361f7e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:12.601493 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aad9b50-997c-4776-a3c7-84bc8361f7e0 None admin] PATH_INFO: `/v3/projects/2a4e4930fe3247d3b04ad23013176d45` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:12.601949 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb9f346-7aad-4b94-8dfd-41463fa2f793 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 15:46:12.602620 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aad9b50-997c-4776-a3c7-84bc8361f7e0 None admin] RBAC: Authorizing `identity:get_project(project_id=2a4e4930fe3247d3b04ad23013176d45)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:12.604147 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aad9b50-997c-4776-a3c7-84bc8361f7e0 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:12.606852 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-7eb9f346-7aad-4b94-8dfd-41463fa2f793 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 30 15:46:12.606852 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:46:12.606852 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:46:12.606852 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:46:12.606852 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:46:12.606852 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:46:12.606852 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:46:12.606852 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:46:12.606852 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:46:12.606852 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:46:12.606852 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:46:12.606852 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:46:12.606852 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 15:46:12.606852 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 15:46:12.606852 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 15:46:12.606852 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 15:46:12.607762 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 15:46:12.607762 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 15:46:12.607762 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 15:46:12.607762 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 15:46:12.607762 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:46:12.607762 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:46:12.607762 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:46:12.607762 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:46:12.607762 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:46:12.607762 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:46:12.607762 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:46:12.607762 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Aug 30 15:46:12.607762 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:46:12.607762 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:46:12.607762 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:46:12.607762 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:46:12.608568 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:46:12.608568 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:46:12.608568 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:46:12.608568 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:46:12.608568 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 15:46:12.608568 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 15:46:12.608568 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 15:46:12.608568 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 15:46:12.608568 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 15:46:12.608568 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 15:46:12.608568 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 30 15:46:12.608568 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Aug 30 15:46:12.609351 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 177/346] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Aug 30 15:46:12 2022] GET /identity/v3/projects/2a4e4930fe3247d3b04ad23013176d45 => generated 298 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:46:12.609419 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 170/347] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Aug 30 15:46:12 2022] GET /identity/v3/roles/member => generated 84 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 15:46:12.609496 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aad9b50-997c-4776-a3c7-84bc8361f7e0 None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:12.609564 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ef795e-dc9a-4673-9fb0-a9d50c019a97 None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:12.609629 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ef795e-dc9a-4673-9fb0-a9d50c019a97 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:12.610392 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ef795e-dc9a-4673-9fb0-a9d50c019a97 None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:12.612029 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 171/348] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 30 15:46:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 15:46:12.615186 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ef795e-dc9a-4673-9fb0-a9d50c019a97 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:12.741851 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aad9b50-997c-4776-a3c7-84bc8361f7e0 None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:12.768439 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ef795e-dc9a-4673-9fb0-a9d50c019a97 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:12.791716 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf3de08-0a3a-401c-b559-76b4b58279ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:12.792702 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf3de08-0a3a-401c-b559-76b4b58279ff None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:12.792977 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf3de08-0a3a-401c-b559-76b4b58279ff None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:12.793296 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf3de08-0a3a-401c-b559-76b4b58279ff None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:12.810749 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca93e5ac-c28e-4424-b618-85016b53abff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:12.811393 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca93e5ac-c28e-4424-b618-85016b53abff None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:12.811665 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca93e5ac-c28e-4424-b618-85016b53abff None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:12.812017 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca93e5ac-c28e-4424-b618-85016b53abff None admin] PATH_INFO: `/v3/groups/d81821a5b2324613b898718d6fb82e2d` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:12.820133 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca93e5ac-c28e-4424-b618-85016b53abff None admin] RBAC: Authorizing `identity:get_group(group_id=d81821a5b2324613b898718d6fb82e2d)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:12.822838 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca93e5ac-c28e-4424-b618-85016b53abff None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:12.824816 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 172/349] 10.222.0.20 () {56 vars in 1180 bytes} [Tue Aug 30 15:46:12 2022] GET /identity/v3/groups/d81821a5b2324613b898718d6fb82e2d => generated 226 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:46:12.825693 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca93e5ac-c28e-4424-b618-85016b53abff None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:12.831001 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:12.878467 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf3de08-0a3a-401c-b559-76b4b58279ff None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:12.880079 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf3de08-0a3a-401c-b559-76b4b58279ff None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:12.880186 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:12.956190 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca93e5ac-c28e-4424-b618-85016b53abff None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:13.007856 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c99a037-0e1f-486f-a646-8140c6e52373 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:13.008609 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c99a037-0e1f-486f-a646-8140c6e52373 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:13.008885 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c99a037-0e1f-486f-a646-8140c6e52373 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:13.009154 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c99a037-0e1f-486f-a646-8140c6e52373 None admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:13.009922 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c99a037-0e1f-486f-a646-8140c6e52373 None admin] RBAC: Adding query filter params (name=member) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:46:13.010219 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c99a037-0e1f-486f-a646-8140c6e52373 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:13.011595 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c99a037-0e1f-486f-a646-8140c6e52373 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:13.012134 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 178/350] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Aug 30 15:46:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 15:46:13.013300 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bca5673-903e-4dc8-8b6a-e56c23427c4d None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:13.013591 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bca5673-903e-4dc8-8b6a-e56c23427c4d None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:13.013864 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bca5673-903e-4dc8-8b6a-e56c23427c4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:13.021237 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 173/351] 10.222.0.20 () {56 vars in 1101 bytes} [Tue Aug 30 15:46:12 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:46:13.022343 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8424aba-fb6e-451d-8e3c-79031242c5fd None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:13.022631 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8424aba-fb6e-451d-8e3c-79031242c5fd None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:13.022896 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8424aba-fb6e-451d-8e3c-79031242c5fd None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:13.024237 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 174/352] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 30 15:46:13 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 15:46:13.024765 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8424aba-fb6e-451d-8e3c-79031242c5fd None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:13.037668 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-6bca5673-903e-4dc8-8b6a-e56c23427c4d None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:46:13.039561 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-6bca5673-903e-4dc8-8b6a-e56c23427c4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:46:13.107840 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:13.125630 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 179/353] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 30 15:46:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 15:46:13.126319 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bca5673-903e-4dc8-8b6a-e56c23427c4d None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:13.143841 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8424aba-fb6e-451d-8e3c-79031242c5fd None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:13.153631 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:13.180876 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1cce4c-267f-4215-a398-9378465ff251 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:13.181553 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1cce4c-267f-4215-a398-9378465ff251 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:13.181823 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1cce4c-267f-4215-a398-9378465ff251 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:13.182086 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1cce4c-267f-4215-a398-9378465ff251 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:13.182871 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1cce4c-267f-4215-a398-9378465ff251 None admin] RBAC: Adding query filter params (group.id=d81821a5b2324613b898718d6fb82e2d, role.id=df8f9f844b1b439181d75a07c1a93418, scope.project.id=2a4e4930fe3247d3b04ad23013176d45) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:46:13.183170 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1cce4c-267f-4215-a398-9378465ff251 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:13.184769 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1cce4c-267f-4215-a398-9378465ff251 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:13.190946 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 175/354] 10.222.0.20 () {56 vars in 1376 bytes} [Tue Aug 30 15:46:13 2022] GET /identity/v3/role_assignments?group.id=d81821a5b2324613b898718d6fb82e2d&scope.project.id=2a4e4930fe3247d3b04ad23013176d45&role.id=df8f9f844b1b439181d75a07c1a93418 => generated 260 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:46:13.191779 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1cce4c-267f-4215-a398-9378465ff251 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:13.316697 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bca5673-903e-4dc8-8b6a-e56c23427c4d None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:13.341585 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1cce4c-267f-4215-a398-9378465ff251 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:13.377462 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958b7983-0c8f-4428-8dc7-c335fd27f066 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:13.378035 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958b7983-0c8f-4428-8dc7-c335fd27f066 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:13.378308 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958b7983-0c8f-4428-8dc7-c335fd27f066 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:13.378573 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958b7983-0c8f-4428-8dc7-c335fd27f066 None admin] PATH_INFO: `/v3/projects/2a4e4930fe3247d3b04ad23013176d45` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:13.380110 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958b7983-0c8f-4428-8dc7-c335fd27f066 None admin] RBAC: Authorizing `identity:get_project(project_id=2a4e4930fe3247d3b04ad23013176d45)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:13.381712 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958b7983-0c8f-4428-8dc7-c335fd27f066 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:13.383936 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 180/355] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Aug 30 15:46:13 2022] GET /identity/v3/projects/2a4e4930fe3247d3b04ad23013176d45 => generated 298 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:46:13.385433 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa1450f-0b17-4206-b909-4376524b4538 None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:13.385714 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa1450f-0b17-4206-b909-4376524b4538 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:13.386002 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa1450f-0b17-4206-b909-4376524b4538 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:13.390686 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2176bcb5-148a-4c4b-a85d-c6d49468d88a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:13.391511 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2176bcb5-148a-4c4b-a85d-c6d49468d88a None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:13.392045 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2176bcb5-148a-4c4b-a85d-c6d49468d88a None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:13.392458 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2176bcb5-148a-4c4b-a85d-c6d49468d88a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:13.405077 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-1aa1450f-0b17-4206-b909-4376524b4538 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:46:13.406594 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-1aa1450f-0b17-4206-b909-4376524b4538 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:46:13.447724 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:13.479783 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:13.482010 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2176bcb5-148a-4c4b-a85d-c6d49468d88a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:13.483515 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2176bcb5-148a-4c4b-a85d-c6d49468d88a None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:13.503927 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 181/356] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 30 15:46:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 15:46:13.505695 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7284d3c7-17ba-4f6d-91b4-fccc406fa370 None None] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:13.505982 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7284d3c7-17ba-4f6d-91b4-fccc406fa370 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:13.506524 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7284d3c7-17ba-4f6d-91b4-fccc406fa370 None None] PATH_INFO: `/` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:13.507538 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 182/357] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 30 15:46:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 15:46:13.508243 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7284d3c7-17ba-4f6d-91b4-fccc406fa370 None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:13.601408 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 176/358] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Aug 30 15:46:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 15:46:13.602744 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2176bcb5-148a-4c4b-a85d-c6d49468d88a None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:13.646595 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7284d3c7-17ba-4f6d-91b4-fccc406fa370 None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:13.682030 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6152dabe-0ad2-49de-a93c-0fa1899cb1ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:13.682665 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6152dabe-0ad2-49de-a93c-0fa1899cb1ad None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:13.682945 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6152dabe-0ad2-49de-a93c-0fa1899cb1ad None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:13.683211 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6152dabe-0ad2-49de-a93c-0fa1899cb1ad None admin] PATH_INFO: `/v3/groups/d81821a5b2324613b898718d6fb82e2d` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:13.684316 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6152dabe-0ad2-49de-a93c-0fa1899cb1ad None admin] RBAC: Authorizing `identity:get_group(group_id=d81821a5b2324613b898718d6fb82e2d)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:13.685840 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6152dabe-0ad2-49de-a93c-0fa1899cb1ad None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:13.687493 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 183/359] 10.222.0.20 () {56 vars in 1180 bytes} [Tue Aug 30 15:46:13 2022] GET /identity/v3/groups/d81821a5b2324613b898718d6fb82e2d => generated 226 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:46:13.688601 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defb276d-5158-4070-a923-44580c69636a None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:13.688903 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defb276d-5158-4070-a923-44580c69636a None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:13.689166 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defb276d-5158-4070-a923-44580c69636a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:13.697640 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:13.708940 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-defb276d-5158-4070-a923-44580c69636a None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:46:13.710140 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-defb276d-5158-4070-a923-44580c69636a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:46:13.731341 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2176bcb5-148a-4c4b-a85d-c6d49468d88a None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:13.769777 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e36d5f-dc09-4c95-9ecd-92d8d14de58e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:13.771717 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:13.772579 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e36d5f-dc09-4c95-9ecd-92d8d14de58e None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:13.773053 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e36d5f-dc09-4c95-9ecd-92d8d14de58e None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:13.773620 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e36d5f-dc09-4c95-9ecd-92d8d14de58e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:13.787185 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e36d5f-dc09-4c95-9ecd-92d8d14de58e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 15:46:13.789701 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 184/360] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 30 15:46:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 15:46:13.790554 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defb276d-5158-4070-a923-44580c69636a None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:13.792905 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-44e36d5f-dc09-4c95-9ecd-92d8d14de58e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 30 15:46:13.792905 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:46:13.792905 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:46:13.792905 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:46:13.792905 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:46:13.792905 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:46:13.792905 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:46:13.792905 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:46:13.792905 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:46:13.792905 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:46:13.792905 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:46:13.792905 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:46:13.792905 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 15:46:13.792905 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 15:46:13.792905 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 15:46:13.792905 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 15:46:13.793936 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 15:46:13.793936 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 15:46:13.793936 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 15:46:13.793936 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 15:46:13.793936 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:46:13.793936 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:46:13.793936 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:46:13.793936 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:46:13.793936 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:46:13.793936 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:46:13.793936 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:46:13.793936 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Aug 30 15:46:13.793936 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:46:13.793936 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:46:13.793936 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:46:13.793936 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:46:13.794731 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:46:13.794731 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:46:13.794731 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:46:13.794731 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:46:13.794731 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 15:46:13.794731 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 15:46:13.794731 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 15:46:13.794731 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 15:46:13.794731 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 15:46:13.794731 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 15:46:13.794731 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 30 15:46:13.794731 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Aug 30 15:46:13.797401 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 177/361] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Aug 30 15:46:13 2022] GET /identity/v3/roles/reader => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 15:46:13.798444 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e36d5f-dc09-4c95-9ecd-92d8d14de58e None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:13.912713 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defb276d-5158-4070-a923-44580c69636a None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:13.930455 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e36d5f-dc09-4c95-9ecd-92d8d14de58e None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:13.947581 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb471e0f-9ec1-4198-a9d0-a8aceb859385 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:13.948220 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb471e0f-9ec1-4198-a9d0-a8aceb859385 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:13.948496 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb471e0f-9ec1-4198-a9d0-a8aceb859385 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:13.948767 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb471e0f-9ec1-4198-a9d0-a8aceb859385 None admin] PATH_INFO: `/v3/projects/c3ff9ff008b04f21a0d68698cf06e6dc` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:13.949924 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb471e0f-9ec1-4198-a9d0-a8aceb859385 None admin] RBAC: Authorizing `identity:get_project(project_id=c3ff9ff008b04f21a0d68698cf06e6dc)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:13.951371 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb471e0f-9ec1-4198-a9d0-a8aceb859385 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:13.953080 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 185/362] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Aug 30 15:46:13 2022] GET /identity/v3/projects/c3ff9ff008b04f21a0d68698cf06e6dc => generated 302 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:46:13.953883 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb471e0f-9ec1-4198-a9d0-a8aceb859385 None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:13.965688 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:13.974243 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb836e97-5d7f-4c02-8c09-9e5a2a6f3bf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:13.975232 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb836e97-5d7f-4c02-8c09-9e5a2a6f3bf7 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:13.975695 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb836e97-5d7f-4c02-8c09-9e5a2a6f3bf7 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:13.976167 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb836e97-5d7f-4c02-8c09-9e5a2a6f3bf7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:13.977186 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb836e97-5d7f-4c02-8c09-9e5a2a6f3bf7 None admin] RBAC: Adding query filter params (group.id=d81821a5b2324613b898718d6fb82e2d, role.id=6b4385c3b449456d848c9abb16766d25, scope.project.id=2a4e4930fe3247d3b04ad23013176d45) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:46:13.978652 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb836e97-5d7f-4c02-8c09-9e5a2a6f3bf7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:13.980316 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb836e97-5d7f-4c02-8c09-9e5a2a6f3bf7 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:13.986797 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 178/363] 10.222.0.20 () {56 vars in 1376 bytes} [Tue Aug 30 15:46:13 2022] GET /identity/v3/role_assignments?group.id=d81821a5b2324613b898718d6fb82e2d&scope.project.id=2a4e4930fe3247d3b04ad23013176d45&role.id=6b4385c3b449456d848c9abb16766d25 => generated 260 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:46:13.987851 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb836e97-5d7f-4c02-8c09-9e5a2a6f3bf7 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:14.054442 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:14.120411 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb836e97-5d7f-4c02-8c09-9e5a2a6f3bf7 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:14.158368 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb74dfd-9812-40e6-bc74-99154b1153e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:14.159081 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb74dfd-9812-40e6-bc74-99154b1153e3 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:14.159363 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb74dfd-9812-40e6-bc74-99154b1153e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:14.159635 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb74dfd-9812-40e6-bc74-99154b1153e3 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:14.160356 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb471e0f-9ec1-4198-a9d0-a8aceb859385 None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:14.171963 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb74dfd-9812-40e6-bc74-99154b1153e3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 15:46:14.177793 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-feb74dfd-9812-40e6-bc74-99154b1153e3 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 15:46:14.177793 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:46:14.177793 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:46:14.177793 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:46:14.177793 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:46:14.177793 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:46:14.177793 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:46:14.177793 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:46:14.177793 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:46:14.177793 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:46:14.177793 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:46:14.177793 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:46:14.177793 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 15:46:14.177793 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 15:46:14.177793 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 15:46:14.177793 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 15:46:14.178748 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 15:46:14.178748 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 15:46:14.178748 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 15:46:14.178748 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 15:46:14.178748 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:46:14.178748 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:46:14.178748 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:46:14.178748 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:46:14.178748 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:46:14.178748 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:46:14.178748 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:46:14.178748 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Aug 30 15:46:14.178748 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:46:14.178748 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:46:14.178748 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:46:14.178748 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:46:14.180108 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:46:14.180108 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:46:14.180108 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:46:14.180108 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:46:14.180108 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 15:46:14.180108 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 15:46:14.180108 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 15:46:14.180108 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 15:46:14.180108 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 15:46:14.180108 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 15:46:14.180108 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 15:46:14.180108 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Aug 30 15:46:14.180108 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 179/364] 10.222.0.20 () {56 vars in 1114 bytes} [Tue Aug 30 15:46:13 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 15:46:14.180108 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb74dfd-9812-40e6-bc74-99154b1153e3 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:14.206305 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5789c36f-1e86-4455-a678-502019f9b836 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:14.207304 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5789c36f-1e86-4455-a678-502019f9b836 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:14.207779 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5789c36f-1e86-4455-a678-502019f9b836 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:14.208316 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5789c36f-1e86-4455-a678-502019f9b836 None admin] PATH_INFO: `/v3/roles` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:14.209176 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5789c36f-1e86-4455-a678-502019f9b836 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81512) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:46:14.209770 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5789c36f-1e86-4455-a678-502019f9b836 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:14.211428 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5789c36f-1e86-4455-a678-502019f9b836 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:14.222923 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 186/365] 10.222.0.20 () {56 vars in 1101 bytes} [Tue Aug 30 15:46:13 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:46:14.224350 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5789c36f-1e86-4455-a678-502019f9b836 None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:14.240198 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:14.289527 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb74dfd-9812-40e6-bc74-99154b1153e3 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:14.327289 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6d363e-4b0b-4bab-a83e-c59631df1d5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:14.328208 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6d363e-4b0b-4bab-a83e-c59631df1d5a None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:14.328631 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6d363e-4b0b-4bab-a83e-c59631df1d5a None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:14.329129 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6d363e-4b0b-4bab-a83e-c59631df1d5a None admin] PATH_INFO: `/v3/groups/d81821a5b2324613b898718d6fb82e2d` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:14.330583 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6d363e-4b0b-4bab-a83e-c59631df1d5a None admin] RBAC: Authorizing `identity:get_group(group_id=d81821a5b2324613b898718d6fb82e2d)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:14.332187 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6d363e-4b0b-4bab-a83e-c59631df1d5a None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:14.334092 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5789c36f-1e86-4455-a678-502019f9b836 None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:14.334826 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 180/366] 10.222.0.20 () {56 vars in 1180 bytes} [Tue Aug 30 15:46:14 2022] GET /identity/v3/groups/d81821a5b2324613b898718d6fb82e2d => generated 226 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:46:14.335837 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6d363e-4b0b-4bab-a83e-c59631df1d5a None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:14.352839 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:14.373125 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4347a4c7-9880-4a48-aceb-04268abb1f85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:14.373958 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4347a4c7-9880-4a48-aceb-04268abb1f85 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:14.374356 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4347a4c7-9880-4a48-aceb-04268abb1f85 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:14.374751 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4347a4c7-9880-4a48-aceb-04268abb1f85 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:14.387220 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4347a4c7-9880-4a48-aceb-04268abb1f85 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 15:46:14.392783 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application [None req-4347a4c7-9880-4a48-aceb-04268abb1f85 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 15:46:14.392783 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:46:14.392783 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:46:14.392783 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:46:14.392783 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:46:14.392783 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:46:14.392783 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:46:14.392783 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:46:14.392783 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:46:14.392783 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:46:14.392783 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:46:14.392783 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:46:14.392783 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 15:46:14.392783 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 15:46:14.392783 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 15:46:14.392783 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 15:46:14.393747 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 15:46:14.393747 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 15:46:14.393747 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 15:46:14.393747 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 15:46:14.393747 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:46:14.393747 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:46:14.393747 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:46:14.393747 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:46:14.393747 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:46:14.393747 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:46:14.393747 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:46:14.393747 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application with Lock( Aug 30 15:46:14.393747 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:46:14.393747 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:46:14.393747 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:46:14.393747 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:46:14.394716 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:46:14.394716 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:46:14.394716 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:46:14.394716 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:46:14.394716 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 15:46:14.394716 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 15:46:14.394716 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 15:46:14.394716 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 15:46:14.394716 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 15:46:14.394716 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 15:46:14.394716 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 15:46:14.394716 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Aug 30 15:46:14.396690 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 187/367] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Aug 30 15:46:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 15:46:14.397648 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4347a4c7-9880-4a48-aceb-04268abb1f85 None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:14.464811 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6d363e-4b0b-4bab-a83e-c59631df1d5a None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:14.504635 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5180d7c-f940-4f5c-af70-dd9cccc48a3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:14.505311 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4347a4c7-9880-4a48-aceb-04268abb1f85 None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:14.505767 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5180d7c-f940-4f5c-af70-dd9cccc48a3f None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:14.506171 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5180d7c-f940-4f5c-af70-dd9cccc48a3f None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:14.506592 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5180d7c-f940-4f5c-af70-dd9cccc48a3f None admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:14.507453 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5180d7c-f940-4f5c-af70-dd9cccc48a3f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:46:14.507927 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5180d7c-f940-4f5c-af70-dd9cccc48a3f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:14.509630 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5180d7c-f940-4f5c-af70-dd9cccc48a3f None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:14.519379 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 181/368] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Aug 30 15:46:14 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:46:14.519938 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:14.528471 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5180d7c-f940-4f5c-af70-dd9cccc48a3f None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:14.542903 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b545a28-ead3-4035-a42d-e2eddd18e86c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:14.543634 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b545a28-ead3-4035-a42d-e2eddd18e86c None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:14.543973 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b545a28-ead3-4035-a42d-e2eddd18e86c None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:14.544302 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b545a28-ead3-4035-a42d-e2eddd18e86c None admin] PATH_INFO: `/v3/users/48da79d776614839bcc9aab9cf039c7b` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:14.545535 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b545a28-ead3-4035-a42d-e2eddd18e86c None admin] RBAC: Authorizing `identity:get_user(user_id=48da79d776614839bcc9aab9cf039c7b)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:14.547169 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b545a28-ead3-4035-a42d-e2eddd18e86c None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:14.548928 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 188/369] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Aug 30 15:46:14 2022] GET /identity/v3/users/48da79d776614839bcc9aab9cf039c7b => generated 293 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:46:14.549725 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b545a28-ead3-4035-a42d-e2eddd18e86c None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:14.610386 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:14.688625 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5180d7c-f940-4f5c-af70-dd9cccc48a3f None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:14.690600 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b545a28-ead3-4035-a42d-e2eddd18e86c None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:14.726919 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac42c9ce-29c3-42f9-8088-db76ba9e579a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:14.727629 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac42c9ce-29c3-42f9-8088-db76ba9e579a None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:14.727999 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac42c9ce-29c3-42f9-8088-db76ba9e579a None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:14.728373 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac42c9ce-29c3-42f9-8088-db76ba9e579a None admin] PATH_INFO: `/v3/roles` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:14.729138 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac42c9ce-29c3-42f9-8088-db76ba9e579a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81512) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:46:14.729611 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac42c9ce-29c3-42f9-8088-db76ba9e579a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:14.731144 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac42c9ce-29c3-42f9-8088-db76ba9e579a None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:14.731783 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f19db0-73c3-4526-944a-585a267b07cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:14.732575 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f19db0-73c3-4526-944a-585a267b07cd None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:14.733139 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f19db0-73c3-4526-944a-585a267b07cd None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:14.733576 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f19db0-73c3-4526-944a-585a267b07cd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:14.734857 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f19db0-73c3-4526-944a-585a267b07cd None admin] RBAC: Adding query filter params (group.id=d81821a5b2324613b898718d6fb82e2d, role.id=df8f9f844b1b439181d75a07c1a93418, scope.project.id=c3ff9ff008b04f21a0d68698cf06e6dc) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:46:14.735310 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f19db0-73c3-4526-944a-585a267b07cd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:14.736911 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f19db0-73c3-4526-944a-585a267b07cd None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:14.744186 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 189/370] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Aug 30 15:46:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:46:14.745190 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac42c9ce-29c3-42f9-8088-db76ba9e579a None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:14.750438 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 182/371] 10.222.0.20 () {56 vars in 1376 bytes} [Tue Aug 30 15:46:14 2022] GET /identity/v3/role_assignments?group.id=d81821a5b2324613b898718d6fb82e2d&scope.project.id=c3ff9ff008b04f21a0d68698cf06e6dc&role.id=df8f9f844b1b439181d75a07c1a93418 => generated 260 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:46:14.751686 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f19db0-73c3-4526-944a-585a267b07cd None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:14.858731 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:14.941249 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac42c9ce-29c3-42f9-8088-db76ba9e579a None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:14.950677 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:14.960646 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f19db0-73c3-4526-944a-585a267b07cd None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:15.001474 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647ea7ea-9e30-407c-9ec2-740f34d3056a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:15.002006 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647ea7ea-9e30-407c-9ec2-740f34d3056a None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:15.002278 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647ea7ea-9e30-407c-9ec2-740f34d3056a None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:15.002541 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647ea7ea-9e30-407c-9ec2-740f34d3056a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:15.027080 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647f07ea-9300-428f-8a3a-06af1022f649 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:15.027699 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647f07ea-9300-428f-8a3a-06af1022f649 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:15.027969 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647f07ea-9300-428f-8a3a-06af1022f649 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:15.028239 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647f07ea-9300-428f-8a3a-06af1022f649 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:15.028965 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647f07ea-9300-428f-8a3a-06af1022f649 None admin] RBAC: Adding query filter params (role.id=5d293e67c8e747e6910e95cf65e56633, user.id=48da79d776614839bcc9aab9cf039c7b, scope.system=all) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:46:15.029298 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647f07ea-9300-428f-8a3a-06af1022f649 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:15.030792 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647f07ea-9300-428f-8a3a-06af1022f649 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:15.056266 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 183/372] 10.222.0.20 () {56 vars in 1308 bytes} [Tue Aug 30 15:46:14 2022] GET /identity/v3/role_assignments?user.id=48da79d776614839bcc9aab9cf039c7b&scope.system=all&role.id=5d293e67c8e747e6910e95cf65e56633 => generated 510 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:46:15.057964 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647f07ea-9300-428f-8a3a-06af1022f649 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:15.110293 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647ea7ea-9e30-407c-9ec2-740f34d3056a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:15.111800 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647ea7ea-9e30-407c-9ec2-740f34d3056a None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:15.268082 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 190/373] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Aug 30 15:46:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 523 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 15:46:15.277440 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647ea7ea-9e30-407c-9ec2-740f34d3056a None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:15.281773 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647f07ea-9300-428f-8a3a-06af1022f649 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:15.305203 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:15.309606 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:15.371648 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dceac25-9b83-4455-a271-b979cd74512f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:15.372272 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dceac25-9b83-4455-a271-b979cd74512f None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:15.372549 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dceac25-9b83-4455-a271-b979cd74512f None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:15.372812 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dceac25-9b83-4455-a271-b979cd74512f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:15.423106 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647ea7ea-9e30-407c-9ec2-740f34d3056a None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:15.479339 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dceac25-9b83-4455-a271-b979cd74512f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:15.480828 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dceac25-9b83-4455-a271-b979cd74512f None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:15.491920 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8953bab-80c0-438b-b2ea-0a999b00af98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:15.492583 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8953bab-80c0-438b-b2ea-0a999b00af98 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:15.492863 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8953bab-80c0-438b-b2ea-0a999b00af98 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:15.493137 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8953bab-80c0-438b-b2ea-0a999b00af98 None admin] PATH_INFO: `/v3/projects/c3ff9ff008b04f21a0d68698cf06e6dc` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:15.494339 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8953bab-80c0-438b-b2ea-0a999b00af98 None admin] RBAC: Authorizing `identity:get_project(project_id=c3ff9ff008b04f21a0d68698cf06e6dc)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:15.495819 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8953bab-80c0-438b-b2ea-0a999b00af98 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:15.500463 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 191/374] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Aug 30 15:46:15 2022] GET /identity/v3/projects/c3ff9ff008b04f21a0d68698cf06e6dc => generated 302 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 15:46:15.510454 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8953bab-80c0-438b-b2ea-0a999b00af98 None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:15.613917 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 184/375] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Aug 30 15:46:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 556 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 15:46:15.620973 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dceac25-9b83-4455-a271-b979cd74512f None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:15.635627 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:15.661377 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:15.688062 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8953bab-80c0-438b-b2ea-0a999b00af98 None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:15.725945 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aff2cf6-c04a-49fc-9b67-9a8af1a0916a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:15.726586 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aff2cf6-c04a-49fc-9b67-9a8af1a0916a None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:15.726865 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aff2cf6-c04a-49fc-9b67-9a8af1a0916a None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:15.727141 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aff2cf6-c04a-49fc-9b67-9a8af1a0916a None admin] PATH_INFO: `/v3/groups/d81821a5b2324613b898718d6fb82e2d` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:15.729211 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aff2cf6-c04a-49fc-9b67-9a8af1a0916a None admin] RBAC: Authorizing `identity:get_group(group_id=d81821a5b2324613b898718d6fb82e2d)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:15.730684 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aff2cf6-c04a-49fc-9b67-9a8af1a0916a None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:15.732438 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 192/376] 10.222.0.20 () {56 vars in 1180 bytes} [Tue Aug 30 15:46:15 2022] GET /identity/v3/groups/d81821a5b2324613b898718d6fb82e2d => generated 226 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 15:46:15.733416 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778ba62a-66a1-4a9f-90a9-e9550e89c244 None None] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:15.733720 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778ba62a-66a1-4a9f-90a9-e9550e89c244 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:15.734045 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778ba62a-66a1-4a9f-90a9-e9550e89c244 None None] PATH_INFO: `/` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:15.735173 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 193/377] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 30 15:46:15 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 15:46:15.742985 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778ba62a-66a1-4a9f-90a9-e9550e89c244 None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:15.791697 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dceac25-9b83-4455-a271-b979cd74512f None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:15.841149 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10997924-f27c-4ff3-8df5-64a305982e5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:15.842516 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10997924-f27c-4ff3-8df5-64a305982e5f None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:15.842995 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10997924-f27c-4ff3-8df5-64a305982e5f None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:15.843513 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10997924-f27c-4ff3-8df5-64a305982e5f None admin] PATH_INFO: `/v3/projects/f6954fa5b9a94855887d7068fc788d39` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:15.845396 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10997924-f27c-4ff3-8df5-64a305982e5f None admin] RBAC: Authorizing `identity:get_project(project_id=f6954fa5b9a94855887d7068fc788d39)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:15.847601 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10997924-f27c-4ff3-8df5-64a305982e5f None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:15.849866 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 185/378] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Aug 30 15:46:15 2022] GET /identity/v3/projects/f6954fa5b9a94855887d7068fc788d39 => generated 344 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 15:46:15.851427 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4d6b0a-e555-4aa7-b77c-0c490595b1af None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:15.851941 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4d6b0a-e555-4aa7-b77c-0c490595b1af None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:15.852338 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4d6b0a-e555-4aa7-b77c-0c490595b1af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:15.874782 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-1d4d6b0a-e555-4aa7-b77c-0c490595b1af None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:46:15.876105 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778ba62a-66a1-4a9f-90a9-e9550e89c244 None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:15.876969 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-1d4d6b0a-e555-4aa7-b77c-0c490595b1af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:46:15.928015 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5f8bee-7911-4e4f-9d99-34c73cde47d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:15.928636 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5f8bee-7911-4e4f-9d99-34c73cde47d1 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:15.928913 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5f8bee-7911-4e4f-9d99-34c73cde47d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:15.929181 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5f8bee-7911-4e4f-9d99-34c73cde47d1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:15.929964 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5f8bee-7911-4e4f-9d99-34c73cde47d1 None admin] RBAC: Adding query filter params (group.id=d81821a5b2324613b898718d6fb82e2d, role.id=6b4385c3b449456d848c9abb16766d25, scope.project.id=c3ff9ff008b04f21a0d68698cf06e6dc) {{(pid=81512) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:46:15.930274 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5f8bee-7911-4e4f-9d99-34c73cde47d1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:15.931719 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5f8bee-7911-4e4f-9d99-34c73cde47d1 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:15.944322 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 194/379] 10.222.0.20 () {56 vars in 1376 bytes} [Tue Aug 30 15:46:15 2022] GET /identity/v3/role_assignments?group.id=d81821a5b2324613b898718d6fb82e2d&scope.project.id=c3ff9ff008b04f21a0d68698cf06e6dc&role.id=6b4385c3b449456d848c9abb16766d25 => generated 260 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 15:46:15.945587 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5f8bee-7911-4e4f-9d99-34c73cde47d1 None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:15.958391 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:16.005878 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 186/380] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 30 15:46:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 15:46:16.088798 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5f8bee-7911-4e4f-9d99-34c73cde47d1 None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:16.129275 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce0f929-dd2f-4832-8316-bc899957bcdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:16.129997 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce0f929-dd2f-4832-8316-bc899957bcdf None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:16.130278 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce0f929-dd2f-4832-8316-bc899957bcdf None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:16.130568 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce0f929-dd2f-4832-8316-bc899957bcdf None admin] PATH_INFO: `/v3/groups/c3fd52f76e9a4aecbb83b91b3ade0b16` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:16.136916 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce0f929-dd2f-4832-8316-bc899957bcdf None admin] RBAC: Authorizing `identity:get_group(group_id=c3fd52f76e9a4aecbb83b91b3ade0b16)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:16.138614 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce0f929-dd2f-4832-8316-bc899957bcdf None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:16.140398 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 195/381] 10.222.0.20 () {56 vars in 1180 bytes} [Tue Aug 30 15:46:15 2022] GET /identity/v3/groups/c3fd52f76e9a4aecbb83b91b3ade0b16 => generated 229 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:46:16.146466 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4d6b0a-e555-4aa7-b77c-0c490595b1af None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:16.155311 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:16.264169 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4d6b0a-e555-4aa7-b77c-0c490595b1af None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:16.311319 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdcd135-c27b-42e5-b440-70f5aa4c58b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:16.311982 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdcd135-c27b-42e5-b440-70f5aa4c58b1 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:16.312258 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdcd135-c27b-42e5-b440-70f5aa4c58b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:16.312528 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdcd135-c27b-42e5-b440-70f5aa4c58b1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:16.313288 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fdcd135-c27b-42e5-b440-70f5aa4c58b1 None admin] RBAC: Adding query filter params (group.id=c3fd52f76e9a4aecbb83b91b3ade0b16, role.id=f265386950a641ac8413b53eda4ae34d, scope.project.id=f6954fa5b9a94855887d7068fc788d39) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:46:16.313599 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fdcd135-c27b-42e5-b440-70f5aa4c58b1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:16.315050 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fdcd135-c27b-42e5-b440-70f5aa4c58b1 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:16.325808 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 187/382] 10.222.0.20 () {56 vars in 1376 bytes} [Tue Aug 30 15:46:16 2022] GET /identity/v3/role_assignments?group.id=c3fd52f76e9a4aecbb83b91b3ade0b16&scope.project.id=f6954fa5b9a94855887d7068fc788d39&role.id=f265386950a641ac8413b53eda4ae34d => generated 260 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:46:16.446185 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce0f929-dd2f-4832-8316-bc899957bcdf None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:16.529210 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:16.567868 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce0f929-dd2f-4832-8316-bc899957bcdf None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:16.599485 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1099c78-25ec-4d4f-96a4-cf2fa5040d0b None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:16.599770 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1099c78-25ec-4d4f-96a4-cf2fa5040d0b None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:16.600044 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1099c78-25ec-4d4f-96a4-cf2fa5040d0b None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:16.601737 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 188/383] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 30 15:46:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 15:46:16.610538 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389d4917-4ea8-4388-ab4c-77ff5a819ff7 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:16.610802 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389d4917-4ea8-4388-ab4c-77ff5a819ff7 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:16.611084 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389d4917-4ea8-4388-ab4c-77ff5a819ff7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:16.613399 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2dc5083-349e-4d0f-a9a0-e2a1a737410a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:16.614386 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dc5083-349e-4d0f-a9a0-e2a1a737410a None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:16.614846 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dc5083-349e-4d0f-a9a0-e2a1a737410a None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:16.615251 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dc5083-349e-4d0f-a9a0-e2a1a737410a None admin] PATH_INFO: `/v3/roles/member` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:16.626909 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2dc5083-349e-4d0f-a9a0-e2a1a737410a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 15:46:16.632687 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application [None req-d2dc5083-349e-4d0f-a9a0-e2a1a737410a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 30 15:46:16.632687 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:46:16.632687 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:46:16.632687 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:46:16.632687 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:46:16.632687 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:46:16.632687 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:46:16.632687 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:46:16.632687 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:46:16.632687 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:46:16.632687 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:46:16.632687 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:46:16.632687 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 15:46:16.632687 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 15:46:16.632687 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 15:46:16.632687 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 15:46:16.633694 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 15:46:16.633694 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 15:46:16.633694 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 15:46:16.633694 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 15:46:16.633694 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:46:16.633694 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:46:16.633694 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:46:16.633694 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:46:16.633694 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:46:16.633694 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:46:16.633694 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:46:16.633694 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application with Lock( Aug 30 15:46:16.633694 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:46:16.633694 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:46:16.633694 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:46:16.633694 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:46:16.634596 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:46:16.634596 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:46:16.634596 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:46:16.634596 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:46:16.634596 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 15:46:16.634596 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 15:46:16.634596 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 15:46:16.634596 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 15:46:16.634596 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 15:46:16.634596 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 15:46:16.634596 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 30 15:46:16.634596 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Aug 30 15:46:16.636867 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 196/384] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Aug 30 15:46:16 2022] GET /identity/v3/roles/member => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 15:46:16.642514 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2dc5083-349e-4d0f-a9a0-e2a1a737410a None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:16.650275 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-389d4917-4ea8-4388-ab4c-77ff5a819ff7 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:46:16.651504 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-389d4917-4ea8-4388-ab4c-77ff5a819ff7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:46:16.757188 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2dc5083-349e-4d0f-a9a0-e2a1a737410a None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:16.789003 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:16.795787 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2d29f6-b943-407c-b810-42a1c2fc748a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:16.796730 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2d29f6-b943-407c-b810-42a1c2fc748a None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:16.797409 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2d29f6-b943-407c-b810-42a1c2fc748a None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:16.797828 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2d29f6-b943-407c-b810-42a1c2fc748a None admin] PATH_INFO: `/v3/roles` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:16.798980 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2d29f6-b943-407c-b810-42a1c2fc748a None admin] RBAC: Adding query filter params (name=member) {{(pid=81512) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:46:16.799426 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2d29f6-b943-407c-b810-42a1c2fc748a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:16.801184 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2d29f6-b943-407c-b810-42a1c2fc748a None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:16.811615 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 197/385] 10.222.0.20 () {56 vars in 1101 bytes} [Tue Aug 30 15:46:16 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:46:16.817389 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2d29f6-b943-407c-b810-42a1c2fc748a None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:16.862831 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:16.878574 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 189/386] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 30 15:46:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 273 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 15:46:16.938851 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2d29f6-b943-407c-b810-42a1c2fc748a None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:16.990899 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f279bc1-e2be-4576-8f11-aa8f70107514 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:16.991535 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f279bc1-e2be-4576-8f11-aa8f70107514 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:16.991814 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f279bc1-e2be-4576-8f11-aa8f70107514 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:16.992085 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f279bc1-e2be-4576-8f11-aa8f70107514 None admin] PATH_INFO: `/v3/groups/d81821a5b2324613b898718d6fb82e2d` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:16.993192 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f279bc1-e2be-4576-8f11-aa8f70107514 None admin] RBAC: Authorizing `identity:get_group(group_id=d81821a5b2324613b898718d6fb82e2d)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:16.994768 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f279bc1-e2be-4576-8f11-aa8f70107514 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:16.996698 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 198/387] 10.222.0.20 () {56 vars in 1180 bytes} [Tue Aug 30 15:46:16 2022] GET /identity/v3/groups/d81821a5b2324613b898718d6fb82e2d => generated 226 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 15:46:17.002475 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389d4917-4ea8-4388-ab4c-77ff5a819ff7 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:17.123923 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389d4917-4ea8-4388-ab4c-77ff5a819ff7 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:17.166132 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4138f5e9-76e9-42fd-8ec7-b61d4990688b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:17.166784 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4138f5e9-76e9-42fd-8ec7-b61d4990688b None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:17.167063 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4138f5e9-76e9-42fd-8ec7-b61d4990688b None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:17.167335 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4138f5e9-76e9-42fd-8ec7-b61d4990688b None admin] PATH_INFO: `/v3/projects/2a4e4930fe3247d3b04ad23013176d45` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:17.173918 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4138f5e9-76e9-42fd-8ec7-b61d4990688b None admin] RBAC: Authorizing `identity:get_project(project_id=2a4e4930fe3247d3b04ad23013176d45)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:17.175450 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4138f5e9-76e9-42fd-8ec7-b61d4990688b None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:17.177454 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 190/388] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Aug 30 15:46:17 2022] GET /identity/v3/projects/2a4e4930fe3247d3b04ad23013176d45 => generated 298 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:46:17.183054 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f279bc1-e2be-4576-8f11-aa8f70107514 None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:17.285586 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:17.308530 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4138f5e9-76e9-42fd-8ec7-b61d4990688b None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:17.363965 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f279bc1-e2be-4576-8f11-aa8f70107514 None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:17.420709 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35262ee2-07e6-41f2-9673-5d7ed59600ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:17.421485 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35262ee2-07e6-41f2-9673-5d7ed59600ac None admin] REQUEST_METHOD: `PUT` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:17.421759 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35262ee2-07e6-41f2-9673-5d7ed59600ac None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:17.422075 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35262ee2-07e6-41f2-9673-5d7ed59600ac None admin] PATH_INFO: `/v3/projects/2a4e4930fe3247d3b04ad23013176d45/groups/d81821a5b2324613b898718d6fb82e2d/roles/df8f9f844b1b439181d75a07c1a93418` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:17.424220 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35262ee2-07e6-41f2-9673-5d7ed59600ac None admin] RBAC: Authorizing `identity:create_grant(project_id=2a4e4930fe3247d3b04ad23013176d45, group_id=d81821a5b2324613b898718d6fb82e2d, role_id=df8f9f844b1b439181d75a07c1a93418)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:17.425718 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35262ee2-07e6-41f2-9673-5d7ed59600ac None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:17.435222 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:17.452143 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 199/389] 10.222.0.20 () {58 vars in 1442 bytes} [Tue Aug 30 15:46:17 2022] PUT /identity/v3/projects/2a4e4930fe3247d3b04ad23013176d45/groups/d81821a5b2324613b898718d6fb82e2d/roles/df8f9f844b1b439181d75a07c1a93418 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:46:17.453847 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acea8e7-0d0e-4973-89be-3ff082d9cbad None None] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:17.454179 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acea8e7-0d0e-4973-89be-3ff082d9cbad None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:17.454496 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acea8e7-0d0e-4973-89be-3ff082d9cbad None None] PATH_INFO: `/` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:17.455883 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 200/390] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 30 15:46:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 15:46:17.462000 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8bbab4-ffe0-4bb0-9da7-0cc4adcf5542 None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:17.462305 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8bbab4-ffe0-4bb0-9da7-0cc4adcf5542 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:17.462539 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8bbab4-ffe0-4bb0-9da7-0cc4adcf5542 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:17.478845 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4138f5e9-76e9-42fd-8ec7-b61d4990688b None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:17.489042 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-6a8bbab4-ffe0-4bb0-9da7-0cc4adcf5542 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:46:17.489042 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-6a8bbab4-ffe0-4bb0-9da7-0cc4adcf5542 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:46:17.524835 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4616a52-2b62-4f0e-9293-833575e30182 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:17.525539 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4616a52-2b62-4f0e-9293-833575e30182 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:17.525820 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4616a52-2b62-4f0e-9293-833575e30182 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:17.526086 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4616a52-2b62-4f0e-9293-833575e30182 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:17.537464 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4616a52-2b62-4f0e-9293-833575e30182 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 15:46:17.542533 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-c4616a52-2b62-4f0e-9293-833575e30182 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 15:46:17.542533 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:46:17.542533 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:46:17.542533 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:46:17.542533 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:46:17.542533 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:46:17.542533 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:46:17.542533 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:46:17.542533 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:46:17.542533 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:46:17.542533 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:46:17.542533 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:46:17.542533 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 15:46:17.542533 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 15:46:17.542533 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 15:46:17.542533 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 15:46:17.543466 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 15:46:17.543466 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 15:46:17.543466 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 15:46:17.543466 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 15:46:17.543466 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:46:17.543466 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:46:17.543466 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:46:17.543466 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:46:17.543466 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:46:17.543466 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:46:17.543466 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:46:17.543466 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Aug 30 15:46:17.543466 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:46:17.543466 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:46:17.543466 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:46:17.543466 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:46:17.544273 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:46:17.544273 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:46:17.544273 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:46:17.544273 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:46:17.544273 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 15:46:17.544273 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 15:46:17.544273 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 15:46:17.544273 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 15:46:17.544273 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 15:46:17.544273 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 15:46:17.544273 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 15:46:17.544273 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Aug 30 15:46:17.544273 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 191/391] 10.222.0.20 () {56 vars in 1114 bytes} [Tue Aug 30 15:46:17 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 15:46:17.549677 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4616a52-2b62-4f0e-9293-833575e30182 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:17.576243 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 201/392] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 30 15:46:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 15:46:17.664096 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4616a52-2b62-4f0e-9293-833575e30182 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:17.692001 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:17.700406 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931666ac-babb-4724-8d52-ca04a4f57b20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:17.701445 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931666ac-babb-4724-8d52-ca04a4f57b20 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:17.702085 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931666ac-babb-4724-8d52-ca04a4f57b20 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:17.702352 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931666ac-babb-4724-8d52-ca04a4f57b20 None admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:17.703084 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931666ac-babb-4724-8d52-ca04a4f57b20 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:46:17.703387 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931666ac-babb-4724-8d52-ca04a4f57b20 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:17.704769 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931666ac-babb-4724-8d52-ca04a4f57b20 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:17.714032 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 192/393] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Aug 30 15:46:17 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:46:17.719957 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8bbab4-ffe0-4bb0-9da7-0cc4adcf5542 None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:17.783675 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:17.850380 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8bbab4-ffe0-4bb0-9da7-0cc4adcf5542 None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:17.900475 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891077fd-b192-4390-9130-e80d2e25f696 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:17.901544 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891077fd-b192-4390-9130-e80d2e25f696 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:17.902039 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891077fd-b192-4390-9130-e80d2e25f696 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:17.902506 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891077fd-b192-4390-9130-e80d2e25f696 None admin] PATH_INFO: `/v3/groups/d81821a5b2324613b898718d6fb82e2d` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:17.903826 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891077fd-b192-4390-9130-e80d2e25f696 None admin] RBAC: Authorizing `identity:get_group(group_id=d81821a5b2324613b898718d6fb82e2d)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:17.905630 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891077fd-b192-4390-9130-e80d2e25f696 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:17.907432 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 202/394] 10.222.0.20 () {56 vars in 1180 bytes} [Tue Aug 30 15:46:17 2022] GET /identity/v3/groups/d81821a5b2324613b898718d6fb82e2d => generated 226 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:46:17.909336 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931666ac-babb-4724-8d52-ca04a4f57b20 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:17.914902 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891077fd-b192-4390-9130-e80d2e25f696 None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:18.047899 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891077fd-b192-4390-9130-e80d2e25f696 None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:18.069461 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:18.084053 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931666ac-babb-4724-8d52-ca04a4f57b20 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:18.102367 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78652a6b-b515-4bb5-b19f-7928f6241165 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:18.103008 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78652a6b-b515-4bb5-b19f-7928f6241165 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:18.103283 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78652a6b-b515-4bb5-b19f-7928f6241165 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:18.103580 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78652a6b-b515-4bb5-b19f-7928f6241165 None admin] PATH_INFO: `/v3/projects/2a4e4930fe3247d3b04ad23013176d45` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:18.105511 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78652a6b-b515-4bb5-b19f-7928f6241165 None admin] RBAC: Authorizing `identity:get_project(project_id=2a4e4930fe3247d3b04ad23013176d45)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:18.106975 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78652a6b-b515-4bb5-b19f-7928f6241165 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:18.108668 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 203/395] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Aug 30 15:46:17 2022] GET /identity/v3/projects/2a4e4930fe3247d3b04ad23013176d45 => generated 298 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:46:18.115375 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78652a6b-b515-4bb5-b19f-7928f6241165 None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:18.141430 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6dbb1c-ddc4-4f64-aa46-5dbbb322c545 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:18.142089 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6dbb1c-ddc4-4f64-aa46-5dbbb322c545 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:18.142366 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6dbb1c-ddc4-4f64-aa46-5dbbb322c545 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:18.142633 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6dbb1c-ddc4-4f64-aa46-5dbbb322c545 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:18.163640 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6dbb1c-ddc4-4f64-aa46-5dbbb322c545 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 15:46:18.169001 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-4d6dbb1c-ddc4-4f64-aa46-5dbbb322c545 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 30 15:46:18.169001 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:46:18.169001 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:46:18.169001 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:46:18.169001 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:46:18.169001 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:46:18.169001 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:46:18.169001 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:46:18.169001 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:46:18.169001 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:46:18.169001 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:46:18.169001 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:46:18.169001 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 15:46:18.169001 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 15:46:18.169001 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 15:46:18.169001 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 15:46:18.169972 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 15:46:18.169972 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 15:46:18.169972 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 15:46:18.169972 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 15:46:18.169972 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:46:18.169972 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:46:18.169972 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:46:18.169972 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:46:18.169972 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:46:18.169972 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:46:18.169972 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:46:18.169972 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Aug 30 15:46:18.169972 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:46:18.169972 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:46:18.169972 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:46:18.169972 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:46:18.170806 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:46:18.170806 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:46:18.170806 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:46:18.170806 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:46:18.170806 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 15:46:18.170806 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 15:46:18.170806 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 15:46:18.170806 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 15:46:18.170806 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 15:46:18.170806 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 15:46:18.170806 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 30 15:46:18.170806 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Aug 30 15:46:18.170806 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 193/396] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Aug 30 15:46:17 2022] GET /identity/v3/roles/member => generated 84 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 15:46:18.177357 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6dbb1c-ddc4-4f64-aa46-5dbbb322c545 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:18.290394 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:18.300312 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78652a6b-b515-4bb5-b19f-7928f6241165 None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:18.343666 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52904298-2c2c-427c-9dd6-a575e61d928b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:18.344342 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52904298-2c2c-427c-9dd6-a575e61d928b None admin] REQUEST_METHOD: `PUT` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:18.344617 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52904298-2c2c-427c-9dd6-a575e61d928b None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:18.344886 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52904298-2c2c-427c-9dd6-a575e61d928b None admin] PATH_INFO: `/v3/projects/2a4e4930fe3247d3b04ad23013176d45/groups/d81821a5b2324613b898718d6fb82e2d/roles/6b4385c3b449456d848c9abb16766d25` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:18.347337 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52904298-2c2c-427c-9dd6-a575e61d928b None admin] RBAC: Authorizing `identity:create_grant(project_id=2a4e4930fe3247d3b04ad23013176d45, group_id=d81821a5b2324613b898718d6fb82e2d, role_id=6b4385c3b449456d848c9abb16766d25)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:18.348801 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52904298-2c2c-427c-9dd6-a575e61d928b None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:18.374753 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 204/397] 10.222.0.20 () {58 vars in 1442 bytes} [Tue Aug 30 15:46:18 2022] PUT /identity/v3/projects/2a4e4930fe3247d3b04ad23013176d45/groups/d81821a5b2324613b898718d6fb82e2d/roles/6b4385c3b449456d848c9abb16766d25 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:46:18.374753 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ca4b9d-94ed-47b9-842c-108479c05d4f None None] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:18.374753 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ca4b9d-94ed-47b9-842c-108479c05d4f None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:18.374753 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ca4b9d-94ed-47b9-842c-108479c05d4f None None] PATH_INFO: `/` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:18.375505 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 205/398] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 30 15:46:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 15:46:18.375750 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6dbb1c-ddc4-4f64-aa46-5dbbb322c545 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:18.381568 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a49b82-211e-4fc0-9165-e3705c8415b3 None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:18.381936 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a49b82-211e-4fc0-9165-e3705c8415b3 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:18.382214 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a49b82-211e-4fc0-9165-e3705c8415b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:18.406578 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-d3a49b82-211e-4fc0-9165-e3705c8415b3 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:46:18.408023 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-d3a49b82-211e-4fc0-9165-e3705c8415b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:46:18.465170 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf6f7ce-870a-46a1-a6c4-a90ab2ce0a72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:18.466232 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf6f7ce-870a-46a1-a6c4-a90ab2ce0a72 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:18.466713 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf6f7ce-870a-46a1-a6c4-a90ab2ce0a72 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:18.467174 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf6f7ce-870a-46a1-a6c4-a90ab2ce0a72 None admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:18.468186 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf6f7ce-870a-46a1-a6c4-a90ab2ce0a72 None admin] RBAC: Adding query filter params (name=member) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:46:18.468686 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf6f7ce-870a-46a1-a6c4-a90ab2ce0a72 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:18.470416 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf6f7ce-870a-46a1-a6c4-a90ab2ce0a72 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:18.477649 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:18.490010 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 194/399] 10.222.0.20 () {56 vars in 1101 bytes} [Tue Aug 30 15:46:18 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:46:18.491873 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 206/400] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 30 15:46:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 15:46:18.500880 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf6f7ce-870a-46a1-a6c4-a90ab2ce0a72 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:18.647951 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf6f7ce-870a-46a1-a6c4-a90ab2ce0a72 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:18.666926 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:18.690805 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5681ee8-3592-487f-a791-df9b89f69602 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:18.691447 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5681ee8-3592-487f-a791-df9b89f69602 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:18.691719 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5681ee8-3592-487f-a791-df9b89f69602 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:18.691984 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5681ee8-3592-487f-a791-df9b89f69602 None admin] PATH_INFO: `/v3/groups/d81821a5b2324613b898718d6fb82e2d` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:18.693208 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5681ee8-3592-487f-a791-df9b89f69602 None admin] RBAC: Authorizing `identity:get_group(group_id=d81821a5b2324613b898718d6fb82e2d)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:18.694846 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5681ee8-3592-487f-a791-df9b89f69602 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:18.696475 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 195/401] 10.222.0.20 () {56 vars in 1180 bytes} [Tue Aug 30 15:46:18 2022] GET /identity/v3/groups/d81821a5b2324613b898718d6fb82e2d => generated 226 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:46:18.702220 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a49b82-211e-4fc0-9165-e3705c8415b3 None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:18.805503 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a49b82-211e-4fc0-9165-e3705c8415b3 None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:18.811505 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2acb65f-758d-4eae-b43b-544e9fd4e571 None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:18.811922 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2acb65f-758d-4eae-b43b-544e9fd4e571 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:18.812271 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2acb65f-758d-4eae-b43b-544e9fd4e571 None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:18.813960 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 196/402] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 30 15:46:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 15:46:18.819500 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6b0c68-eaa8-43fa-bafe-4aff5e59acda None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:18.819850 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6b0c68-eaa8-43fa-bafe-4aff5e59acda None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:18.820193 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6b0c68-eaa8-43fa-bafe-4aff5e59acda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:18.839433 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-4c6b0c68-eaa8-43fa-bafe-4aff5e59acda None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:46:18.840702 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-4c6b0c68-eaa8-43fa-bafe-4aff5e59acda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:46:18.842784 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f129ccb2-e09b-4d25-9c2d-24bab3e48ebf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:18.843480 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f129ccb2-e09b-4d25-9c2d-24bab3e48ebf None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:18.843752 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f129ccb2-e09b-4d25-9c2d-24bab3e48ebf None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:18.844017 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f129ccb2-e09b-4d25-9c2d-24bab3e48ebf None admin] PATH_INFO: `/v3/projects/c3ff9ff008b04f21a0d68698cf06e6dc` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:18.845230 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f129ccb2-e09b-4d25-9c2d-24bab3e48ebf None admin] RBAC: Authorizing `identity:get_project(project_id=c3ff9ff008b04f21a0d68698cf06e6dc)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:18.846678 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f129ccb2-e09b-4d25-9c2d-24bab3e48ebf None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:18.853173 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 207/403] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Aug 30 15:46:18 2022] GET /identity/v3/projects/c3ff9ff008b04f21a0d68698cf06e6dc => generated 302 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:46:18.854309 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f129ccb2-e09b-4d25-9c2d-24bab3e48ebf None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:18.903657 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:18.918315 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 197/404] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 30 15:46:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 15:46:18.919311 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6b0c68-eaa8-43fa-bafe-4aff5e59acda None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:18.963634 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f129ccb2-e09b-4d25-9c2d-24bab3e48ebf None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:19.001847 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7c5d6f-177f-4c36-b3dd-7db1b24f71df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:19.002512 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7c5d6f-177f-4c36-b3dd-7db1b24f71df None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:19.002785 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7c5d6f-177f-4c36-b3dd-7db1b24f71df None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:19.003059 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7c5d6f-177f-4c36-b3dd-7db1b24f71df None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:19.014527 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7c5d6f-177f-4c36-b3dd-7db1b24f71df None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 15:46:19.019677 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application [None req-0f7c5d6f-177f-4c36-b3dd-7db1b24f71df None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 15:46:19.019677 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:46:19.019677 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:46:19.019677 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:46:19.019677 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:46:19.019677 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:46:19.019677 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:46:19.019677 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:46:19.019677 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:46:19.019677 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:46:19.019677 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:46:19.019677 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:46:19.019677 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 15:46:19.019677 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 15:46:19.019677 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 15:46:19.019677 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 15:46:19.020594 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 15:46:19.020594 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 15:46:19.020594 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 15:46:19.020594 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 15:46:19.020594 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:46:19.020594 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:46:19.020594 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:46:19.020594 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:46:19.020594 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:46:19.020594 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:46:19.020594 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:46:19.020594 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application with Lock( Aug 30 15:46:19.020594 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:46:19.020594 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:46:19.020594 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:46:19.020594 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:46:19.022403 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:46:19.022403 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:46:19.022403 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:46:19.022403 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:46:19.022403 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 15:46:19.022403 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 15:46:19.022403 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 15:46:19.022403 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 15:46:19.022403 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 15:46:19.022403 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 15:46:19.022403 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 15:46:19.022403 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Aug 30 15:46:19.022403 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 208/405] 10.222.0.20 () {56 vars in 1114 bytes} [Tue Aug 30 15:46:18 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 15:46:19.027187 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6b0c68-eaa8-43fa-bafe-4aff5e59acda None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:19.028344 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7c5d6f-177f-4c36-b3dd-7db1b24f71df None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:19.045274 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:19.073125 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178f17eb-6645-44f5-be28-af880f01ae61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:19.074163 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178f17eb-6645-44f5-be28-af880f01ae61 None admin] REQUEST_METHOD: `PUT` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:19.074721 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178f17eb-6645-44f5-be28-af880f01ae61 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:19.075258 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178f17eb-6645-44f5-be28-af880f01ae61 None admin] PATH_INFO: `/v3/projects/c3ff9ff008b04f21a0d68698cf06e6dc/groups/d81821a5b2324613b898718d6fb82e2d/roles/df8f9f844b1b439181d75a07c1a93418` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:19.077573 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178f17eb-6645-44f5-be28-af880f01ae61 None admin] RBAC: Authorizing `identity:create_grant(project_id=c3ff9ff008b04f21a0d68698cf06e6dc, group_id=d81821a5b2324613b898718d6fb82e2d, role_id=df8f9f844b1b439181d75a07c1a93418)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:19.079697 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178f17eb-6645-44f5-be28-af880f01ae61 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:19.106246 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 198/406] 10.222.0.20 () {58 vars in 1442 bytes} [Tue Aug 30 15:46:18 2022] PUT /identity/v3/projects/c3ff9ff008b04f21a0d68698cf06e6dc/groups/d81821a5b2324613b898718d6fb82e2d/roles/df8f9f844b1b439181d75a07c1a93418 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:46:19.211481 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7c5d6f-177f-4c36-b3dd-7db1b24f71df None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:19.218583 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:19.246933 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178f17eb-6645-44f5-be28-af880f01ae61 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:19.264822 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebad3610-1aa9-4c4f-b878-ffc59a2c7ef9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:19.265591 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebad3610-1aa9-4c4f-b878-ffc59a2c7ef9 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:19.265848 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebad3610-1aa9-4c4f-b878-ffc59a2c7ef9 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:19.266105 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebad3610-1aa9-4c4f-b878-ffc59a2c7ef9 None admin] PATH_INFO: `/v3/roles` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:19.266789 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebad3610-1aa9-4c4f-b878-ffc59a2c7ef9 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81512) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:46:19.267091 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebad3610-1aa9-4c4f-b878-ffc59a2c7ef9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:19.268516 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebad3610-1aa9-4c4f-b878-ffc59a2c7ef9 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:19.285706 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 209/407] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Aug 30 15:46:19 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:46:19.291754 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebad3610-1aa9-4c4f-b878-ffc59a2c7ef9 None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:19.366209 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178f17eb-6645-44f5-be28-af880f01ae61 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:19.413359 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568900a2-d90e-49fb-92eb-9293293be589 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:19.414415 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568900a2-d90e-49fb-92eb-9293293be589 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:19.414901 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568900a2-d90e-49fb-92eb-9293293be589 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:19.415382 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568900a2-d90e-49fb-92eb-9293293be589 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:19.416846 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebad3610-1aa9-4c4f-b878-ffc59a2c7ef9 None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:19.426450 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568900a2-d90e-49fb-92eb-9293293be589 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 15:46:19.431638 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-568900a2-d90e-49fb-92eb-9293293be589 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 15:46:19.431638 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:46:19.431638 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:46:19.431638 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:46:19.431638 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:46:19.431638 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:46:19.431638 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:46:19.431638 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:46:19.431638 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:46:19.431638 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:46:19.431638 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:46:19.431638 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:46:19.431638 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 15:46:19.431638 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 15:46:19.431638 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 15:46:19.431638 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 15:46:19.432511 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 15:46:19.432511 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 15:46:19.432511 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 15:46:19.432511 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 15:46:19.432511 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:46:19.432511 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:46:19.432511 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:46:19.432511 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:46:19.432511 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:46:19.432511 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:46:19.432511 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:46:19.432511 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Aug 30 15:46:19.432511 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:46:19.432511 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:46:19.432511 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:46:19.432511 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:46:19.433341 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:46:19.433341 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:46:19.433341 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:46:19.433341 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:46:19.433341 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 15:46:19.433341 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 15:46:19.433341 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 15:46:19.433341 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 15:46:19.433341 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 15:46:19.433341 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 15:46:19.433341 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 15:46:19.433341 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Aug 30 15:46:19.435572 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 199/408] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Aug 30 15:46:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 15:46:19.446010 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568900a2-d90e-49fb-92eb-9293293be589 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:19.469520 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0bc018-0c8e-4737-8d40-2c7532eed652 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:19.471234 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0bc018-0c8e-4737-8d40-2c7532eed652 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:19.472346 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0bc018-0c8e-4737-8d40-2c7532eed652 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:19.472803 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0bc018-0c8e-4737-8d40-2c7532eed652 None admin] PATH_INFO: `/v3/groups/d81821a5b2324613b898718d6fb82e2d` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:19.475082 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0bc018-0c8e-4737-8d40-2c7532eed652 None admin] RBAC: Authorizing `identity:get_group(group_id=d81821a5b2324613b898718d6fb82e2d)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:19.479315 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0bc018-0c8e-4737-8d40-2c7532eed652 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:19.481845 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 210/409] 10.222.0.20 () {56 vars in 1180 bytes} [Tue Aug 30 15:46:19 2022] GET /identity/v3/groups/d81821a5b2324613b898718d6fb82e2d => generated 226 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 15:46:19.488079 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0bc018-0c8e-4737-8d40-2c7532eed652 None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:19.509940 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:19.551187 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:19.632303 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568900a2-d90e-49fb-92eb-9293293be589 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:19.669030 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0bc018-0c8e-4737-8d40-2c7532eed652 None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:19.695320 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07e4692-0c6a-4e8c-9f01-6196b75a212d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:19.695931 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07e4692-0c6a-4e8c-9f01-6196b75a212d None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:19.696201 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07e4692-0c6a-4e8c-9f01-6196b75a212d None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:19.696464 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07e4692-0c6a-4e8c-9f01-6196b75a212d None admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:19.697131 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07e4692-0c6a-4e8c-9f01-6196b75a212d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:46:19.697455 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07e4692-0c6a-4e8c-9f01-6196b75a212d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:19.698831 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07e4692-0c6a-4e8c-9f01-6196b75a212d None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:19.716799 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 200/410] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Aug 30 15:46:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:46:19.722593 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07e4692-0c6a-4e8c-9f01-6196b75a212d None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:19.735174 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c801c2-0703-4bfe-8749-75c399da7f22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:19.735867 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c801c2-0703-4bfe-8749-75c399da7f22 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:19.736144 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c801c2-0703-4bfe-8749-75c399da7f22 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:19.736412 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c801c2-0703-4bfe-8749-75c399da7f22 None admin] PATH_INFO: `/v3/projects/c3ff9ff008b04f21a0d68698cf06e6dc` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:19.737686 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c801c2-0703-4bfe-8749-75c399da7f22 None admin] RBAC: Authorizing `identity:get_project(project_id=c3ff9ff008b04f21a0d68698cf06e6dc)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:19.741106 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c801c2-0703-4bfe-8749-75c399da7f22 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:19.744025 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 211/411] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Aug 30 15:46:19 2022] GET /identity/v3/projects/c3ff9ff008b04f21a0d68698cf06e6dc => generated 302 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:46:19.748552 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c801c2-0703-4bfe-8749-75c399da7f22 None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:19.876538 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c801c2-0703-4bfe-8749-75c399da7f22 None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:19.878626 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:19.940591 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db923b77-2990-4651-b951-b3c7dc94c82e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:19.942535 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db923b77-2990-4651-b951-b3c7dc94c82e None admin] REQUEST_METHOD: `PUT` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:19.942820 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db923b77-2990-4651-b951-b3c7dc94c82e None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:19.943088 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db923b77-2990-4651-b951-b3c7dc94c82e None admin] PATH_INFO: `/v3/projects/c3ff9ff008b04f21a0d68698cf06e6dc/groups/d81821a5b2324613b898718d6fb82e2d/roles/6b4385c3b449456d848c9abb16766d25` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:19.948663 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db923b77-2990-4651-b951-b3c7dc94c82e None admin] RBAC: Authorizing `identity:create_grant(project_id=c3ff9ff008b04f21a0d68698cf06e6dc, group_id=d81821a5b2324613b898718d6fb82e2d, role_id=6b4385c3b449456d848c9abb16766d25)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:19.949658 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:19.950185 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db923b77-2990-4651-b951-b3c7dc94c82e None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:19.963474 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07e4692-0c6a-4e8c-9f01-6196b75a212d None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:19.976732 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 212/412] 10.222.0.20 () {58 vars in 1442 bytes} [Tue Aug 30 15:46:19 2022] PUT /identity/v3/projects/c3ff9ff008b04f21a0d68698cf06e6dc/groups/d81821a5b2324613b898718d6fb82e2d/roles/6b4385c3b449456d848c9abb16766d25 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 15:46:20.011419 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc33137-e1e0-4217-a9c4-e454a12f6705 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:20.012121 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc33137-e1e0-4217-a9c4-e454a12f6705 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:20.012386 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc33137-e1e0-4217-a9c4-e454a12f6705 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:20.012701 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc33137-e1e0-4217-a9c4-e454a12f6705 None admin] PATH_INFO: `/v3/groups/c3fd52f76e9a4aecbb83b91b3ade0b16` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:20.013872 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc33137-e1e0-4217-a9c4-e454a12f6705 None admin] RBAC: Authorizing `identity:get_group(group_id=c3fd52f76e9a4aecbb83b91b3ade0b16)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:20.015431 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc33137-e1e0-4217-a9c4-e454a12f6705 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:20.017506 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 201/413] 10.222.0.20 () {56 vars in 1180 bytes} [Tue Aug 30 15:46:19 2022] GET /identity/v3/groups/c3fd52f76e9a4aecbb83b91b3ade0b16 => generated 229 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:46:20.026620 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db923b77-2990-4651-b951-b3c7dc94c82e None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:20.104468 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac91397-fe34-4b09-886f-2c0776a57656 None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:20.104795 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac91397-fe34-4b09-886f-2c0776a57656 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:20.105006 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac91397-fe34-4b09-886f-2c0776a57656 None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:20.106454 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 202/414] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 30 15:46:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 15:46:20.112027 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f693c2-e083-46c4-ac23-72570b3299c1 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:20.112313 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f693c2-e083-46c4-ac23-72570b3299c1 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:20.112591 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f693c2-e083-46c4-ac23-72570b3299c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:20.144344 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-54f693c2-e083-46c4-ac23-72570b3299c1 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:46:20.145624 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-54f693c2-e083-46c4-ac23-72570b3299c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:46:20.178298 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db923b77-2990-4651-b951-b3c7dc94c82e None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:20.217091 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154c8404-846b-4c4b-bf77-56145d293720 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:20.217760 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154c8404-846b-4c4b-bf77-56145d293720 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:20.218144 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154c8404-846b-4c4b-bf77-56145d293720 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:20.218416 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154c8404-846b-4c4b-bf77-56145d293720 None admin] PATH_INFO: `/v3/projects/f6954fa5b9a94855887d7068fc788d39` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:20.219734 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154c8404-846b-4c4b-bf77-56145d293720 None admin] RBAC: Authorizing `identity:get_project(project_id=f6954fa5b9a94855887d7068fc788d39)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:20.221757 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154c8404-846b-4c4b-bf77-56145d293720 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:20.223871 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 213/415] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Aug 30 15:46:20 2022] GET /identity/v3/projects/f6954fa5b9a94855887d7068fc788d39 => generated 344 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:46:20.232702 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154c8404-846b-4c4b-bf77-56145d293720 None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:20.248944 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 203/416] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 30 15:46:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 15:46:20.288459 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:20.395297 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154c8404-846b-4c4b-bf77-56145d293720 None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:20.443164 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31423001-1b5e-4fd1-bf0f-7ae326e02282 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:20.443835 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31423001-1b5e-4fd1-bf0f-7ae326e02282 None admin] REQUEST_METHOD: `PUT` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:20.444112 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31423001-1b5e-4fd1-bf0f-7ae326e02282 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:20.444380 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31423001-1b5e-4fd1-bf0f-7ae326e02282 None admin] PATH_INFO: `/v3/projects/f6954fa5b9a94855887d7068fc788d39/groups/c3fd52f76e9a4aecbb83b91b3ade0b16/roles/f265386950a641ac8413b53eda4ae34d` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:20.446389 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31423001-1b5e-4fd1-bf0f-7ae326e02282 None admin] RBAC: Authorizing `identity:create_grant(project_id=f6954fa5b9a94855887d7068fc788d39, group_id=c3fd52f76e9a4aecbb83b91b3ade0b16, role_id=f265386950a641ac8413b53eda4ae34d)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:20.447850 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31423001-1b5e-4fd1-bf0f-7ae326e02282 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:20.474659 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 214/417] 10.222.0.20 () {58 vars in 1442 bytes} [Tue Aug 30 15:46:20 2022] PUT /identity/v3/projects/f6954fa5b9a94855887d7068fc788d39/groups/c3fd52f76e9a4aecbb83b91b3ade0b16/roles/f265386950a641ac8413b53eda4ae34d => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:46:20.647103 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f693c2-e083-46c4-ac23-72570b3299c1 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:20.773618 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:20.788169 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f693c2-e083-46c4-ac23-72570b3299c1 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:20.824533 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b354a6-5dbb-4c8b-89b1-b9151ba2a2ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:20.825174 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b354a6-5dbb-4c8b-89b1-b9151ba2a2ab None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:20.825475 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b354a6-5dbb-4c8b-89b1-b9151ba2a2ab None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:20.825745 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b354a6-5dbb-4c8b-89b1-b9151ba2a2ab None admin] PATH_INFO: `/v3/roles/member` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:20.836236 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b354a6-5dbb-4c8b-89b1-b9151ba2a2ab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 15:46:20.841102 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-d7b354a6-5dbb-4c8b-89b1-b9151ba2a2ab None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 30 15:46:20.841102 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:46:20.841102 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:46:20.841102 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:46:20.841102 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:46:20.841102 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:46:20.841102 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:46:20.841102 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:46:20.841102 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:46:20.841102 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:46:20.841102 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:46:20.841102 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:46:20.841102 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 15:46:20.841102 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 15:46:20.841102 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 15:46:20.841102 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 15:46:20.842021 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 15:46:20.842021 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 15:46:20.842021 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 15:46:20.842021 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 15:46:20.842021 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:46:20.842021 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:46:20.842021 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:46:20.842021 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:46:20.842021 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:46:20.842021 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:46:20.842021 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:46:20.842021 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Aug 30 15:46:20.842021 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:46:20.842021 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:46:20.842021 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:46:20.842021 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:46:20.843254 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:46:20.843254 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:46:20.843254 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:46:20.843254 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:46:20.843254 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 15:46:20.843254 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 15:46:20.843254 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 15:46:20.843254 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 15:46:20.843254 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 15:46:20.843254 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 15:46:20.843254 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 30 15:46:20.843254 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Aug 30 15:46:20.843254 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 204/418] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Aug 30 15:46:20 2022] GET /identity/v3/roles/member => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 15:46:20.848444 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31423001-1b5e-4fd1-bf0f-7ae326e02282 None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:20.850156 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f020abad-c6a9-4134-8147-113d54c381ff None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:20.850440 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f020abad-c6a9-4134-8147-113d54c381ff None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:20.850702 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f020abad-c6a9-4134-8147-113d54c381ff None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:20.852603 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 205/419] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 30 15:46:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 15:46:20.860063 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4527a6c4-706e-4ac1-b202-9fb323ee116a None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:20.860349 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4527a6c4-706e-4ac1-b202-9fb323ee116a None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:20.860620 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4527a6c4-706e-4ac1-b202-9fb323ee116a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:20.895944 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-4527a6c4-706e-4ac1-b202-9fb323ee116a None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:46:20.897063 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-4527a6c4-706e-4ac1-b202-9fb323ee116a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:46:21.007590 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 206/420] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 30 15:46:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 15:46:21.021636 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:21.026473 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31423001-1b5e-4fd1-bf0f-7ae326e02282 None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:21.062524 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e560d0-66af-48af-b076-cdf92c483175 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:21.063472 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e560d0-66af-48af-b076-cdf92c483175 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:21.063858 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e560d0-66af-48af-b076-cdf92c483175 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:21.064268 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e560d0-66af-48af-b076-cdf92c483175 None admin] PATH_INFO: `/v3/roles` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:21.065010 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e560d0-66af-48af-b076-cdf92c483175 None admin] RBAC: Adding query filter params (name=member) {{(pid=81512) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:46:21.065448 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e560d0-66af-48af-b076-cdf92c483175 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:21.067091 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e560d0-66af-48af-b076-cdf92c483175 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:21.075892 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 215/421] 10.222.0.20 () {56 vars in 1101 bytes} [Tue Aug 30 15:46:20 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:46:21.081512 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4527a6c4-706e-4ac1-b202-9fb323ee116a None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:21.199887 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:21.221640 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4527a6c4-706e-4ac1-b202-9fb323ee116a None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:21.266038 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45719c3b-f943-44cd-b8f2-8d4a6b7da472 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:21.266657 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45719c3b-f943-44cd-b8f2-8d4a6b7da472 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:21.266933 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45719c3b-f943-44cd-b8f2-8d4a6b7da472 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:21.267200 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45719c3b-f943-44cd-b8f2-8d4a6b7da472 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:21.323025 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e560d0-66af-48af-b076-cdf92c483175 None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:21.390806 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45719c3b-f943-44cd-b8f2-8d4a6b7da472 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:21.392301 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45719c3b-f943-44cd-b8f2-8d4a6b7da472 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:21.506770 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:21.534886 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 207/422] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Aug 30 15:46:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 454 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 15:46:21.540951 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca913649-a814-4189-99c6-c7e4923b9439 None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:21.541259 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca913649-a814-4189-99c6-c7e4923b9439 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:21.541538 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca913649-a814-4189-99c6-c7e4923b9439 None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:21.543162 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 208/423] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 30 15:46:21 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 15:46:21.543888 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca913649-a814-4189-99c6-c7e4923b9439 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:21.544893 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e560d0-66af-48af-b076-cdf92c483175 None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:21.578992 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea30e4cc-b3f4-4e74-a193-548495ac1d2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:21.579978 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea30e4cc-b3f4-4e74-a193-548495ac1d2a None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:21.580446 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea30e4cc-b3f4-4e74-a193-548495ac1d2a None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:21.580904 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea30e4cc-b3f4-4e74-a193-548495ac1d2a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:21.591380 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea30e4cc-b3f4-4e74-a193-548495ac1d2a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 15:46:21.596914 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application [None req-ea30e4cc-b3f4-4e74-a193-548495ac1d2a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 15:46:21.596914 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:46:21.596914 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:46:21.596914 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:46:21.596914 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:46:21.596914 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:46:21.596914 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:46:21.596914 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:46:21.596914 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:46:21.596914 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:46:21.596914 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:46:21.596914 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:46:21.596914 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 15:46:21.596914 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 15:46:21.596914 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 15:46:21.596914 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 15:46:21.597820 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 15:46:21.597820 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 15:46:21.597820 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 15:46:21.597820 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 15:46:21.597820 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:46:21.597820 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:46:21.597820 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:46:21.597820 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:46:21.597820 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:46:21.597820 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:46:21.597820 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:46:21.597820 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application with Lock( Aug 30 15:46:21.597820 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:46:21.597820 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:46:21.597820 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:46:21.597820 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:46:21.598641 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:46:21.598641 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:46:21.598641 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:46:21.598641 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:46:21.598641 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 15:46:21.598641 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 15:46:21.598641 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 15:46:21.598641 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 15:46:21.598641 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 15:46:21.598641 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 15:46:21.598641 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 15:46:21.598641 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Aug 30 15:46:21.601155 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 216/424] 10.222.0.20 () {56 vars in 1114 bytes} [Tue Aug 30 15:46:21 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 15:46:21.602595 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa8a3c3-5c4f-4e28-9491-f0bb6d7cf3fa None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:21.603083 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa8a3c3-5c4f-4e28-9491-f0bb6d7cf3fa None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:21.603552 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa8a3c3-5c4f-4e28-9491-f0bb6d7cf3fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:21.621685 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-4fa8a3c3-5c4f-4e28-9491-f0bb6d7cf3fa None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:46:21.623151 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-4fa8a3c3-5c4f-4e28-9491-f0bb6d7cf3fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:46:21.636572 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:21.663414 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca913649-a814-4189-99c6-c7e4923b9439 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:21.702077 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c5dbd2-5a9a-44f9-867b-ef19dbdb87f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:21.702743 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c5dbd2-5a9a-44f9-867b-ef19dbdb87f9 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:21.703017 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c5dbd2-5a9a-44f9-867b-ef19dbdb87f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:21.703284 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c5dbd2-5a9a-44f9-867b-ef19dbdb87f9 None admin] PATH_INFO: `/v3/projects/2a4e4930fe3247d3b04ad23013176d45` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:21.704010 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 217/425] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 30 15:46:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 15:46:21.704483 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c5dbd2-5a9a-44f9-867b-ef19dbdb87f9 None admin] RBAC: Authorizing `identity:get_project(project_id=2a4e4930fe3247d3b04ad23013176d45)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:21.705351 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa8a3c3-5c4f-4e28-9491-f0bb6d7cf3fa None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:21.710601 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c5dbd2-5a9a-44f9-867b-ef19dbdb87f9 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:21.712791 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 209/426] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Aug 30 15:46:21 2022] GET /identity/v3/projects/2a4e4930fe3247d3b04ad23013176d45 => generated 298 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:46:21.719037 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c5dbd2-5a9a-44f9-867b-ef19dbdb87f9 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:21.806878 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:21.893824 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa8a3c3-5c4f-4e28-9491-f0bb6d7cf3fa None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:21.900747 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c5dbd2-5a9a-44f9-867b-ef19dbdb87f9 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:21.949546 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e81591a-2d97-4709-aab9-1f44acb8d2ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:21.950337 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e81591a-2d97-4709-aab9-1f44acb8d2ab None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:21.950614 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e81591a-2d97-4709-aab9-1f44acb8d2ab None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:21.950882 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e81591a-2d97-4709-aab9-1f44acb8d2ab None admin] PATH_INFO: `/v3/groups/d81821a5b2324613b898718d6fb82e2d` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:21.952003 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e81591a-2d97-4709-aab9-1f44acb8d2ab None admin] RBAC: Authorizing `identity:get_group(group_id=d81821a5b2324613b898718d6fb82e2d)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:21.953508 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e81591a-2d97-4709-aab9-1f44acb8d2ab None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:21.955448 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 210/427] 10.222.0.20 () {56 vars in 1180 bytes} [Tue Aug 30 15:46:21 2022] GET /identity/v3/groups/d81821a5b2324613b898718d6fb82e2d => generated 226 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:46:21.957553 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8856bf64-bd74-4c5a-817c-9a52639af414 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:21.958174 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8856bf64-bd74-4c5a-817c-9a52639af414 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:21.958456 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8856bf64-bd74-4c5a-817c-9a52639af414 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:21.958724 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8856bf64-bd74-4c5a-817c-9a52639af414 None admin] PATH_INFO: `/v3/roles` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:21.959399 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8856bf64-bd74-4c5a-817c-9a52639af414 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81512) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:46:21.959701 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8856bf64-bd74-4c5a-817c-9a52639af414 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:21.961074 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8856bf64-bd74-4c5a-817c-9a52639af414 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:21.961246 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e81591a-2d97-4709-aab9-1f44acb8d2ab None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:21.980995 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 218/428] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Aug 30 15:46:21 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:46:21.987277 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8856bf64-bd74-4c5a-817c-9a52639af414 None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:21.994613 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:22.081321 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e81591a-2d97-4709-aab9-1f44acb8d2ab None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:22.109600 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:22.128658 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8856bf64-bd74-4c5a-817c-9a52639af414 None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:22.131147 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ef2774-3a7a-444b-b1d6-2db76756ec44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:22.132698 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ef2774-3a7a-444b-b1d6-2db76756ec44 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:22.133042 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ef2774-3a7a-444b-b1d6-2db76756ec44 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:22.133408 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ef2774-3a7a-444b-b1d6-2db76756ec44 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:22.134214 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ef2774-3a7a-444b-b1d6-2db76756ec44 None admin] RBAC: Adding query filter params (group.id=d81821a5b2324613b898718d6fb82e2d, role.id=df8f9f844b1b439181d75a07c1a93418, scope.project.id=2a4e4930fe3247d3b04ad23013176d45) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:46:22.134578 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ef2774-3a7a-444b-b1d6-2db76756ec44 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:22.136090 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ef2774-3a7a-444b-b1d6-2db76756ec44 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:22.145299 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 211/429] 10.222.0.20 () {56 vars in 1376 bytes} [Tue Aug 30 15:46:21 2022] GET /identity/v3/role_assignments?group.id=d81821a5b2324613b898718d6fb82e2d&scope.project.id=2a4e4930fe3247d3b04ad23013176d45&role.id=df8f9f844b1b439181d75a07c1a93418 => generated 611 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:46:22.146311 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ef2774-3a7a-444b-b1d6-2db76756ec44 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:22.191787 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c76ca4-999b-41a8-945d-9f638132e535 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:22.192695 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c76ca4-999b-41a8-945d-9f638132e535 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:22.193111 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c76ca4-999b-41a8-945d-9f638132e535 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:22.193583 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c76ca4-999b-41a8-945d-9f638132e535 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:22.302320 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c76ca4-999b-41a8-945d-9f638132e535 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:22.303820 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c76ca4-999b-41a8-945d-9f638132e535 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:22.306899 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:22.308501 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ef2774-3a7a-444b-b1d6-2db76756ec44 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:22.354391 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9121f84-45ac-4e78-9d41-a1ffce276d8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:22.355120 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9121f84-45ac-4e78-9d41-a1ffce276d8e None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:22.355392 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9121f84-45ac-4e78-9d41-a1ffce276d8e None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:22.356046 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9121f84-45ac-4e78-9d41-a1ffce276d8e None admin] PATH_INFO: `/v3/roles/member` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:22.373932 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9121f84-45ac-4e78-9d41-a1ffce276d8e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 15:46:22.380112 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-f9121f84-45ac-4e78-9d41-a1ffce276d8e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 30 15:46:22.380112 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:46:22.380112 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:46:22.380112 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:46:22.380112 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:46:22.380112 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:46:22.380112 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:46:22.380112 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:46:22.380112 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:46:22.380112 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:46:22.380112 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:46:22.380112 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:46:22.380112 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 15:46:22.380112 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 15:46:22.380112 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 15:46:22.380112 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 15:46:22.381045 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 15:46:22.381045 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 15:46:22.381045 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 15:46:22.381045 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 15:46:22.381045 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:46:22.381045 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:46:22.381045 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:46:22.381045 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:46:22.381045 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:46:22.381045 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:46:22.381045 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:46:22.381045 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Aug 30 15:46:22.381045 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:46:22.381045 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:46:22.381045 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:46:22.381045 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:46:22.381887 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:46:22.381887 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:46:22.381887 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:46:22.381887 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:46:22.381887 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 15:46:22.381887 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 15:46:22.381887 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 15:46:22.381887 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 15:46:22.381887 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 15:46:22.381887 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 15:46:22.381887 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 30 15:46:22.381887 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Aug 30 15:46:22.382495 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 212/430] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Aug 30 15:46:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 15:46:22.383635 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81459312-47e4-4d85-bdb1-9c30390d7887 None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:22.388100 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81459312-47e4-4d85-bdb1-9c30390d7887 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:22.388100 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81459312-47e4-4d85-bdb1-9c30390d7887 None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:22.388100 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 213/431] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 30 15:46:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 15:46:22.388100 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81459312-47e4-4d85-bdb1-9c30390d7887 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:22.433313 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 219/432] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Aug 30 15:46:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 445 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 30 15:46:22.434680 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ff732f-8936-44df-b609-3b0a2de89afd None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:22.434974 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ff732f-8936-44df-b609-3b0a2de89afd None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:22.435247 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ff732f-8936-44df-b609-3b0a2de89afd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:22.445685 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:22.458008 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-90ff732f-8936-44df-b609-3b0a2de89afd None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:46:22.459200 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-90ff732f-8936-44df-b609-3b0a2de89afd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:46:22.520650 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81459312-47e4-4d85-bdb1-9c30390d7887 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:22.549958 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 220/433] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 30 15:46:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 15:46:22.550877 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ff732f-8936-44df-b609-3b0a2de89afd None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:22.562274 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c2bf0d-3d08-41dc-96dd-351c29e91509 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:22.563383 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c2bf0d-3d08-41dc-96dd-351c29e91509 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:22.563858 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c2bf0d-3d08-41dc-96dd-351c29e91509 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:22.564341 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c2bf0d-3d08-41dc-96dd-351c29e91509 None admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:22.565263 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c2bf0d-3d08-41dc-96dd-351c29e91509 None admin] RBAC: Adding query filter params (name=member) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:46:22.565742 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c2bf0d-3d08-41dc-96dd-351c29e91509 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:22.567527 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c2bf0d-3d08-41dc-96dd-351c29e91509 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:22.580154 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 214/434] 10.222.0.20 () {56 vars in 1101 bytes} [Tue Aug 30 15:46:22 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 15:46:22.587793 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c2bf0d-3d08-41dc-96dd-351c29e91509 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:22.593967 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:22.674384 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ff732f-8936-44df-b609-3b0a2de89afd None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:22.724730 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7444e0ea-9d1a-485f-916d-43872aebb433 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:22.726906 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7444e0ea-9d1a-485f-916d-43872aebb433 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:22.727310 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7444e0ea-9d1a-485f-916d-43872aebb433 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:22.727649 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7444e0ea-9d1a-485f-916d-43872aebb433 None admin] PATH_INFO: `/v3/projects/2a4e4930fe3247d3b04ad23013176d45` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:22.728967 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7444e0ea-9d1a-485f-916d-43872aebb433 None admin] RBAC: Authorizing `identity:get_project(project_id=2a4e4930fe3247d3b04ad23013176d45)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:22.731026 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7444e0ea-9d1a-485f-916d-43872aebb433 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:22.733707 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 221/435] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Aug 30 15:46:22 2022] GET /identity/v3/projects/2a4e4930fe3247d3b04ad23013176d45 => generated 298 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:46:22.736652 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c2bf0d-3d08-41dc-96dd-351c29e91509 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:22.739587 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:22.741369 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7444e0ea-9d1a-485f-916d-43872aebb433 None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:22.799633 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ba3c79-84a8-4e29-9905-8c811a07c0b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:22.801491 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ba3c79-84a8-4e29-9905-8c811a07c0b3 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:22.801950 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ba3c79-84a8-4e29-9905-8c811a07c0b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:22.802422 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ba3c79-84a8-4e29-9905-8c811a07c0b3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:22.874692 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7444e0ea-9d1a-485f-916d-43872aebb433 None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:22.876219 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:22.879126 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ba3c79-84a8-4e29-9905-8c811a07c0b3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:22.880649 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ba3c79-84a8-4e29-9905-8c811a07c0b3 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:22.908487 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade23bad-151e-4309-a69d-e4f6442f8492 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:22.909041 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade23bad-151e-4309-a69d-e4f6442f8492 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:22.909343 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade23bad-151e-4309-a69d-e4f6442f8492 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:22.909614 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade23bad-151e-4309-a69d-e4f6442f8492 None admin] PATH_INFO: `/v3/groups/d81821a5b2324613b898718d6fb82e2d` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:22.910719 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade23bad-151e-4309-a69d-e4f6442f8492 None admin] RBAC: Authorizing `identity:get_group(group_id=d81821a5b2324613b898718d6fb82e2d)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:22.912153 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade23bad-151e-4309-a69d-e4f6442f8492 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:22.913760 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 222/436] 10.222.0.20 () {56 vars in 1180 bytes} [Tue Aug 30 15:46:22 2022] GET /identity/v3/groups/d81821a5b2324613b898718d6fb82e2d => generated 226 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:46:22.914866 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216aa21c-054d-415b-bfcd-f912c7a6cd75 None None] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:22.915143 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216aa21c-054d-415b-bfcd-f912c7a6cd75 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:22.915410 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216aa21c-054d-415b-bfcd-f912c7a6cd75 None None] PATH_INFO: `/` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:22.916822 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 223/437] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 30 15:46:22 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 15:46:22.918118 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216aa21c-054d-415b-bfcd-f912c7a6cd75 None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:22.977808 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 215/438] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Aug 30 15:46:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 15:46:22.982844 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ba3c79-84a8-4e29-9905-8c811a07c0b3 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:22.985381 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:23.020033 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216aa21c-054d-415b-bfcd-f912c7a6cd75 None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:23.059083 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315383ef-ae84-463b-97e1-f5a87f956115 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:23.059710 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315383ef-ae84-463b-97e1-f5a87f956115 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:23.059984 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315383ef-ae84-463b-97e1-f5a87f956115 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:23.060428 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315383ef-ae84-463b-97e1-f5a87f956115 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:23.070523 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315383ef-ae84-463b-97e1-f5a87f956115 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 15:46:23.075258 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application [None req-315383ef-ae84-463b-97e1-f5a87f956115 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 15:46:23.075258 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:46:23.075258 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:46:23.075258 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:46:23.075258 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:46:23.075258 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:46:23.075258 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:46:23.075258 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:46:23.075258 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:46:23.075258 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:46:23.075258 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:46:23.075258 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:46:23.075258 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 15:46:23.075258 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 15:46:23.075258 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 15:46:23.075258 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 15:46:23.076137 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 15:46:23.076137 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 15:46:23.076137 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 15:46:23.076137 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 15:46:23.076137 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:46:23.076137 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:46:23.076137 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:46:23.076137 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:46:23.076137 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:46:23.076137 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:46:23.076137 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:46:23.076137 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application with Lock( Aug 30 15:46:23.076137 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:46:23.076137 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:46:23.076137 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:46:23.076137 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:46:23.082905 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:46:23.082905 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:46:23.082905 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:46:23.082905 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:46:23.082905 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 15:46:23.082905 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 15:46:23.082905 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 15:46:23.082905 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 15:46:23.082905 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 15:46:23.082905 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 15:46:23.082905 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 15:46:23.082905 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Aug 30 15:46:23.082905 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 224/439] 10.222.0.20 () {56 vars in 1114 bytes} [Tue Aug 30 15:46:22 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 15:46:23.082905 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0669cc88-4be2-43e5-8848-f8dd8535769c None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:23.083806 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0669cc88-4be2-43e5-8848-f8dd8535769c None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:23.083806 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0669cc88-4be2-43e5-8848-f8dd8535769c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:23.084042 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ba3c79-84a8-4e29-9905-8c811a07c0b3 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:23.098067 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-0669cc88-4be2-43e5-8848-f8dd8535769c None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:46:23.099242 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-0669cc88-4be2-43e5-8848-f8dd8535769c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:46:23.119697 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6cf90d-c251-450b-98e8-af390868166b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:23.120305 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6cf90d-c251-450b-98e8-af390868166b None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:23.120566 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6cf90d-c251-450b-98e8-af390868166b None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:23.120827 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6cf90d-c251-450b-98e8-af390868166b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:23.121925 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6cf90d-c251-450b-98e8-af390868166b None admin] RBAC: Adding query filter params (group.id=d81821a5b2324613b898718d6fb82e2d, role.id=6b4385c3b449456d848c9abb16766d25, scope.project.id=2a4e4930fe3247d3b04ad23013176d45) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:46:23.122226 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6cf90d-c251-450b-98e8-af390868166b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:23.123665 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6cf90d-c251-450b-98e8-af390868166b None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:23.123771 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:23.129487 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 216/440] 10.222.0.20 () {56 vars in 1376 bytes} [Tue Aug 30 15:46:22 2022] GET /identity/v3/role_assignments?group.id=d81821a5b2324613b898718d6fb82e2d&scope.project.id=2a4e4930fe3247d3b04ad23013176d45&role.id=6b4385c3b449456d848c9abb16766d25 => generated 611 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:46:23.130270 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6cf90d-c251-450b-98e8-af390868166b None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:23.183765 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 225/441] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 30 15:46:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 15:46:23.184911 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0669cc88-4be2-43e5-8848-f8dd8535769c None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:23.206778 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:23.247116 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6cf90d-c251-450b-98e8-af390868166b None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:23.289815 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf21ed6-1bad-4152-a9fe-5e7dcb601a42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:23.290874 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf21ed6-1bad-4152-a9fe-5e7dcb601a42 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:23.291342 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf21ed6-1bad-4152-a9fe-5e7dcb601a42 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:23.291829 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf21ed6-1bad-4152-a9fe-5e7dcb601a42 None admin] PATH_INFO: `/v3/projects/c3ff9ff008b04f21a0d68698cf06e6dc` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:23.293034 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf21ed6-1bad-4152-a9fe-5e7dcb601a42 None admin] RBAC: Authorizing `identity:get_project(project_id=c3ff9ff008b04f21a0d68698cf06e6dc)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:23.294577 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf21ed6-1bad-4152-a9fe-5e7dcb601a42 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:23.301048 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 217/442] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Aug 30 15:46:23 2022] GET /identity/v3/projects/c3ff9ff008b04f21a0d68698cf06e6dc => generated 302 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:46:23.301753 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf21ed6-1bad-4152-a9fe-5e7dcb601a42 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:23.318352 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0669cc88-4be2-43e5-8848-f8dd8535769c None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:23.370534 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02748c46-a96b-4fbb-971c-e56b786c17dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:23.372073 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02748c46-a96b-4fbb-971c-e56b786c17dc None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:23.372572 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02748c46-a96b-4fbb-971c-e56b786c17dc None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:23.373068 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02748c46-a96b-4fbb-971c-e56b786c17dc None admin] PATH_INFO: `/v3/roles` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:23.374422 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02748c46-a96b-4fbb-971c-e56b786c17dc None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81512) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:46:23.374982 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02748c46-a96b-4fbb-971c-e56b786c17dc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:23.377735 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02748c46-a96b-4fbb-971c-e56b786c17dc None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:23.393333 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 226/443] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Aug 30 15:46:23 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:46:23.394795 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02748c46-a96b-4fbb-971c-e56b786c17dc None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:23.415296 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf21ed6-1bad-4152-a9fe-5e7dcb601a42 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:23.422520 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:23.466011 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ff7013-d42c-4c6b-9896-5a17bb8c14d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:23.466734 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ff7013-d42c-4c6b-9896-5a17bb8c14d8 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:23.467007 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ff7013-d42c-4c6b-9896-5a17bb8c14d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:23.467273 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ff7013-d42c-4c6b-9896-5a17bb8c14d8 None admin] PATH_INFO: `/v3/groups/d81821a5b2324613b898718d6fb82e2d` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:23.468739 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ff7013-d42c-4c6b-9896-5a17bb8c14d8 None admin] RBAC: Authorizing `identity:get_group(group_id=d81821a5b2324613b898718d6fb82e2d)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:23.470208 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ff7013-d42c-4c6b-9896-5a17bb8c14d8 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:23.472111 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 218/444] 10.222.0.20 () {56 vars in 1180 bytes} [Tue Aug 30 15:46:23 2022] GET /identity/v3/groups/d81821a5b2324613b898718d6fb82e2d => generated 226 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 15:46:23.472727 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ff7013-d42c-4c6b-9896-5a17bb8c14d8 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:23.475782 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:23.518280 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02748c46-a96b-4fbb-971c-e56b786c17dc None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:23.562920 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc80d9ba-984e-4723-a655-6bf70269e752 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:23.563681 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc80d9ba-984e-4723-a655-6bf70269e752 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:23.563961 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc80d9ba-984e-4723-a655-6bf70269e752 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:23.564227 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc80d9ba-984e-4723-a655-6bf70269e752 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:23.582060 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ff7013-d42c-4c6b-9896-5a17bb8c14d8 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:23.614637 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefedb47-55b0-4d5f-ab97-cbc5ca803b93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:23.615258 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefedb47-55b0-4d5f-ab97-cbc5ca803b93 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:23.615530 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefedb47-55b0-4d5f-ab97-cbc5ca803b93 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:23.615795 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefedb47-55b0-4d5f-ab97-cbc5ca803b93 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:23.628382 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefedb47-55b0-4d5f-ab97-cbc5ca803b93 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 15:46:23.631471 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-eefedb47-55b0-4d5f-ab97-cbc5ca803b93 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 15:46:23.631471 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:46:23.631471 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:46:23.631471 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:46:23.631471 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:46:23.631471 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:46:23.631471 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:46:23.631471 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:46:23.631471 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:46:23.631471 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:46:23.631471 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:46:23.631471 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:46:23.631471 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 15:46:23.631471 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 15:46:23.631471 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 15:46:23.631471 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 15:46:23.632336 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 15:46:23.632336 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 15:46:23.632336 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 15:46:23.632336 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 15:46:23.632336 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:46:23.632336 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:46:23.632336 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:46:23.632336 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:46:23.632336 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:46:23.632336 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:46:23.632336 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:46:23.632336 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Aug 30 15:46:23.632336 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:46:23.632336 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:46:23.632336 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:46:23.632336 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:46:23.633151 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:46:23.633151 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:46:23.633151 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:46:23.633151 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:46:23.633151 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 15:46:23.633151 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 15:46:23.633151 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 15:46:23.633151 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 15:46:23.633151 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 15:46:23.633151 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 15:46:23.633151 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 15:46:23.633151 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Aug 30 15:46:23.633795 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc80d9ba-984e-4723-a655-6bf70269e752 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:23.633868 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 219/445] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Aug 30 15:46:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 15:46:23.633941 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc80d9ba-984e-4723-a655-6bf70269e752 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:23.634008 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefedb47-55b0-4d5f-ab97-cbc5ca803b93 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:23.658970 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:23.699112 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:23.739092 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 227/446] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Aug 30 15:46:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 30 15:46:23.739092 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc80d9ba-984e-4723-a655-6bf70269e752 None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:23.739357 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefedb47-55b0-4d5f-ab97-cbc5ca803b93 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:23.771886 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed3dfd0-b857-448d-9788-427619152070 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:23.772520 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed3dfd0-b857-448d-9788-427619152070 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:23.772766 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed3dfd0-b857-448d-9788-427619152070 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:23.773031 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed3dfd0-b857-448d-9788-427619152070 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:23.773875 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed3dfd0-b857-448d-9788-427619152070 None admin] RBAC: Adding query filter params (group.id=d81821a5b2324613b898718d6fb82e2d, role.id=df8f9f844b1b439181d75a07c1a93418, scope.project.id=c3ff9ff008b04f21a0d68698cf06e6dc) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:46:23.774171 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed3dfd0-b857-448d-9788-427619152070 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:23.775596 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed3dfd0-b857-448d-9788-427619152070 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:23.781335 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 220/447] 10.222.0.20 () {56 vars in 1376 bytes} [Tue Aug 30 15:46:23 2022] GET /identity/v3/role_assignments?group.id=d81821a5b2324613b898718d6fb82e2d&scope.project.id=c3ff9ff008b04f21a0d68698cf06e6dc&role.id=df8f9f844b1b439181d75a07c1a93418 => generated 611 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:46:23.782093 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed3dfd0-b857-448d-9788-427619152070 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:23.852194 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc80d9ba-984e-4723-a655-6bf70269e752 None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:23.877794 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:23.890366 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176d45ba-8f6b-4454-8b60-0b09d7d58ac9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:23.891036 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176d45ba-8f6b-4454-8b60-0b09d7d58ac9 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:23.891318 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176d45ba-8f6b-4454-8b60-0b09d7d58ac9 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:23.891583 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176d45ba-8f6b-4454-8b60-0b09d7d58ac9 None admin] PATH_INFO: `/v3/roles` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:23.892254 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176d45ba-8f6b-4454-8b60-0b09d7d58ac9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81512) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:46:23.892553 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176d45ba-8f6b-4454-8b60-0b09d7d58ac9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:23.893994 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176d45ba-8f6b-4454-8b60-0b09d7d58ac9 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:23.895924 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed3dfd0-b857-448d-9788-427619152070 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:23.903308 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 228/448] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Aug 30 15:46:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:46:23.909313 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176d45ba-8f6b-4454-8b60-0b09d7d58ac9 None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:23.928322 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:23.939314 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81c4717-0e19-49f9-bb3d-4aceb8b2e25c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:23.939951 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81c4717-0e19-49f9-bb3d-4aceb8b2e25c None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:23.940912 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81c4717-0e19-49f9-bb3d-4aceb8b2e25c None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:23.940912 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81c4717-0e19-49f9-bb3d-4aceb8b2e25c None admin] PATH_INFO: `/v3/projects/c3ff9ff008b04f21a0d68698cf06e6dc` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:23.941862 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81c4717-0e19-49f9-bb3d-4aceb8b2e25c None admin] RBAC: Authorizing `identity:get_project(project_id=c3ff9ff008b04f21a0d68698cf06e6dc)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:23.943460 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81c4717-0e19-49f9-bb3d-4aceb8b2e25c None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:23.945083 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 221/449] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Aug 30 15:46:23 2022] GET /identity/v3/projects/c3ff9ff008b04f21a0d68698cf06e6dc => generated 302 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:46:23.950979 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81c4717-0e19-49f9-bb3d-4aceb8b2e25c None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:24.027227 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176d45ba-8f6b-4454-8b60-0b09d7d58ac9 None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:24.067311 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e541ce-7b8a-4b21-a61b-1402fc5251e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:24.067811 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81c4717-0e19-49f9-bb3d-4aceb8b2e25c None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:24.067934 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e541ce-7b8a-4b21-a61b-1402fc5251e8 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:24.068177 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e541ce-7b8a-4b21-a61b-1402fc5251e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:24.068451 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e541ce-7b8a-4b21-a61b-1402fc5251e8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:24.112488 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5ed8d6-2d52-4241-8a2a-f5c9e1061c26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:24.113236 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5ed8d6-2d52-4241-8a2a-f5c9e1061c26 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:24.113548 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5ed8d6-2d52-4241-8a2a-f5c9e1061c26 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:24.114196 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5ed8d6-2d52-4241-8a2a-f5c9e1061c26 None admin] PATH_INFO: `/v3/groups/d81821a5b2324613b898718d6fb82e2d` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:24.115012 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5ed8d6-2d52-4241-8a2a-f5c9e1061c26 None admin] RBAC: Authorizing `identity:get_group(group_id=d81821a5b2324613b898718d6fb82e2d)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:24.116446 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5ed8d6-2d52-4241-8a2a-f5c9e1061c26 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:24.121906 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 222/450] 10.222.0.20 () {56 vars in 1180 bytes} [Tue Aug 30 15:46:23 2022] GET /identity/v3/groups/d81821a5b2324613b898718d6fb82e2d => generated 226 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 15:46:24.124672 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5ed8d6-2d52-4241-8a2a-f5c9e1061c26 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:24.129968 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:24.148256 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e541ce-7b8a-4b21-a61b-1402fc5251e8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:24.149795 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e541ce-7b8a-4b21-a61b-1402fc5251e8 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:24.185319 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:24.224129 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5ed8d6-2d52-4241-8a2a-f5c9e1061c26 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:24.245992 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 229/451] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Aug 30 15:46:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 30 15:46:24.252375 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e541ce-7b8a-4b21-a61b-1402fc5251e8 None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:24.257409 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815fe43c-5eaa-49b0-8992-f8ce8f663e73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:24.258020 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815fe43c-5eaa-49b0-8992-f8ce8f663e73 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:24.258298 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815fe43c-5eaa-49b0-8992-f8ce8f663e73 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:24.258560 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815fe43c-5eaa-49b0-8992-f8ce8f663e73 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:24.259282 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815fe43c-5eaa-49b0-8992-f8ce8f663e73 None admin] RBAC: Adding query filter params (group.id=d81821a5b2324613b898718d6fb82e2d, role.id=6b4385c3b449456d848c9abb16766d25, scope.project.id=c3ff9ff008b04f21a0d68698cf06e6dc) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:46:24.259868 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815fe43c-5eaa-49b0-8992-f8ce8f663e73 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:24.261017 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815fe43c-5eaa-49b0-8992-f8ce8f663e73 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:24.266657 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 223/452] 10.222.0.20 () {56 vars in 1376 bytes} [Tue Aug 30 15:46:24 2022] GET /identity/v3/role_assignments?group.id=d81821a5b2324613b898718d6fb82e2d&scope.project.id=c3ff9ff008b04f21a0d68698cf06e6dc&role.id=6b4385c3b449456d848c9abb16766d25 => generated 611 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 15:46:24.339423 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:24.356717 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e541ce-7b8a-4b21-a61b-1402fc5251e8 None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:24.394333 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139a1972-f7af-4fbf-8c08-bed6e8f05e35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:24.394967 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139a1972-f7af-4fbf-8c08-bed6e8f05e35 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:24.395299 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139a1972-f7af-4fbf-8c08-bed6e8f05e35 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:24.395638 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139a1972-f7af-4fbf-8c08-bed6e8f05e35 None admin] PATH_INFO: `/v3/projects/f6954fa5b9a94855887d7068fc788d39` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:24.396842 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139a1972-f7af-4fbf-8c08-bed6e8f05e35 None admin] RBAC: Authorizing `identity:get_project(project_id=f6954fa5b9a94855887d7068fc788d39)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:24.398320 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139a1972-f7af-4fbf-8c08-bed6e8f05e35 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:24.399998 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 230/453] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Aug 30 15:46:24 2022] GET /identity/v3/projects/f6954fa5b9a94855887d7068fc788d39 => generated 344 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 15:46:24.405743 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815fe43c-5eaa-49b0-8992-f8ce8f663e73 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:24.510035 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815fe43c-5eaa-49b0-8992-f8ce8f663e73 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:24.536186 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:24.545525 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d06037-b6f7-4d5d-a2fe-b0542602c3fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:24.546243 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d06037-b6f7-4d5d-a2fe-b0542602c3fd None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:24.546524 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d06037-b6f7-4d5d-a2fe-b0542602c3fd None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:24.546798 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d06037-b6f7-4d5d-a2fe-b0542602c3fd None admin] PATH_INFO: `/v3/groups/c3fd52f76e9a4aecbb83b91b3ade0b16` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:24.547924 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d06037-b6f7-4d5d-a2fe-b0542602c3fd None admin] RBAC: Authorizing `identity:get_group(group_id=c3fd52f76e9a4aecbb83b91b3ade0b16)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:24.549376 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d06037-b6f7-4d5d-a2fe-b0542602c3fd None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:24.550959 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 224/454] 10.222.0.20 () {56 vars in 1180 bytes} [Tue Aug 30 15:46:24 2022] GET /identity/v3/groups/c3fd52f76e9a4aecbb83b91b3ade0b16 => generated 229 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:46:24.557209 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139a1972-f7af-4fbf-8c08-bed6e8f05e35 None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:24.660077 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139a1972-f7af-4fbf-8c08-bed6e8f05e35 None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:24.697359 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813209d3-4205-4410-81c6-2cb60ad6b913 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:24.697947 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813209d3-4205-4410-81c6-2cb60ad6b913 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:24.698218 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813209d3-4205-4410-81c6-2cb60ad6b913 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:24.698479 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813209d3-4205-4410-81c6-2cb60ad6b913 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:24.699191 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813209d3-4205-4410-81c6-2cb60ad6b913 None admin] RBAC: Adding query filter params (group.id=c3fd52f76e9a4aecbb83b91b3ade0b16, role.id=f265386950a641ac8413b53eda4ae34d, scope.project.id=f6954fa5b9a94855887d7068fc788d39) {{(pid=81512) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:46:24.699492 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813209d3-4205-4410-81c6-2cb60ad6b913 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:24.700923 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813209d3-4205-4410-81c6-2cb60ad6b913 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:24.706476 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 231/455] 10.222.0.20 () {56 vars in 1376 bytes} [Tue Aug 30 15:46:24 2022] GET /identity/v3/role_assignments?group.id=c3fd52f76e9a4aecbb83b91b3ade0b16&scope.project.id=f6954fa5b9a94855887d7068fc788d39&role.id=f265386950a641ac8413b53eda4ae34d => generated 611 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:46:27.012334 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c95a4a4-7b03-4336-a39f-2a507bb921d2 None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:27.012860 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c95a4a4-7b03-4336-a39f-2a507bb921d2 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:27.012860 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c95a4a4-7b03-4336-a39f-2a507bb921d2 None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:27.014465 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 225/456] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 30 15:46:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 15:46:27.019376 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b24b05-0cd7-4c19-be0c-41bd5959c640 None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:27.019777 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b24b05-0cd7-4c19-be0c-41bd5959c640 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:27.019964 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b24b05-0cd7-4c19-be0c-41bd5959c640 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:27.041596 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-e2b24b05-0cd7-4c19-be0c-41bd5959c640 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:46:27.042799 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-e2b24b05-0cd7-4c19-be0c-41bd5959c640 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:46:27.058754 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:27.116094 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e1fdb9-48be-4528-b3ed-d87a05e01163 None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:27.116374 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e1fdb9-48be-4528-b3ed-d87a05e01163 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:27.116831 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e1fdb9-48be-4528-b3ed-d87a05e01163 None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:27.118571 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 226/457] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 30 15:46:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 15:46:27.123437 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34a946e-3c43-47c3-aa11-601a5d825492 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:27.123861 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34a946e-3c43-47c3-aa11-601a5d825492 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:27.124329 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34a946e-3c43-47c3-aa11-601a5d825492 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:27.130129 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 232/458] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 30 15:46:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 15:46:27.162259 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-a34a946e-3c43-47c3-aa11-601a5d825492 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:46:27.163705 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-a34a946e-3c43-47c3-aa11-601a5d825492 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:46:27.259792 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 227/459] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 30 15:46:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 15:46:27.387237 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bce91d8-d59c-47d1-86c1-8e84d37ab24d None None] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:27.387771 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bce91d8-d59c-47d1-86c1-8e84d37ab24d None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:27.388184 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bce91d8-d59c-47d1-86c1-8e84d37ab24d None None] PATH_INFO: `/` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:27.390041 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 233/460] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 30 15:46:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 15:46:27.394818 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3032b5-a652-43e1-969f-d163761a73a5 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:27.395100 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3032b5-a652-43e1-969f-d163761a73a5 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:27.395369 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3032b5-a652-43e1-969f-d163761a73a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:27.421607 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-da3032b5-a652-43e1-969f-d163761a73a5 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:46:27.422814 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-da3032b5-a652-43e1-969f-d163761a73a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:46:27.506915 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 228/461] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 30 15:46:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 15:46:27.511522 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bce91d8-d59c-47d1-86c1-8e84d37ab24d None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:27.604042 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3032b5-a652-43e1-969f-d163761a73a5 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:27.661719 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bce91d8-d59c-47d1-86c1-8e84d37ab24d None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:27.696843 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bee3a3-653d-456d-8c1a-ed945ac449a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:27.697503 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bee3a3-653d-456d-8c1a-ed945ac449a5 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:27.697780 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bee3a3-653d-456d-8c1a-ed945ac449a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:27.698049 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bee3a3-653d-456d-8c1a-ed945ac449a5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:27.708735 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72bee3a3-653d-456d-8c1a-ed945ac449a5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:27.710248 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72bee3a3-653d-456d-8c1a-ed945ac449a5 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:27.717387 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:27.724215 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:27.727372 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application [None req-72bee3a3-653d-456d-8c1a-ed945ac449a5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 15:46:27.727372 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:46:27.727372 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:46:27.727372 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:46:27.727372 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:46:27.727372 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:46:27.727372 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:46:27.727372 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:46:27.727372 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:46:27.727372 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:46:27.727372 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:46:27.727372 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:46:27.727372 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 15:46:27.727372 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 15:46:27.727372 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 15:46:27.727372 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 15:46:27.728276 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:46:27.728276 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:46:27.728276 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:46:27.728276 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:46:27.728276 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:46:27.728276 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:46:27.728276 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:46:27.728276 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application with Lock( Aug 30 15:46:27.728276 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:46:27.728276 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:46:27.728276 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:46:27.728276 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:46:27.728276 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:46:27.728276 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:46:27.728276 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:46:27.728276 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:46:27.728276 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 15:46:27.729127 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 15:46:27.729127 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 15:46:27.729127 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Aug 30 15:46:27.729343 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 234/462] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 30 15:46:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 15:46:27.736971 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bee3a3-653d-456d-8c1a-ed945ac449a5 None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:27.816315 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3032b5-a652-43e1-969f-d163761a73a5 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:27.858433 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf1f289-90d3-42e2-b9ad-66bc1080a49a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:27.859106 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf1f289-90d3-42e2-b9ad-66bc1080a49a None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:27.859393 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf1f289-90d3-42e2-b9ad-66bc1080a49a None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:27.859668 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf1f289-90d3-42e2-b9ad-66bc1080a49a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:27.870186 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf1f289-90d3-42e2-b9ad-66bc1080a49a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:27.872003 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf1f289-90d3-42e2-b9ad-66bc1080a49a None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:27.890262 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-aaf1f289-90d3-42e2-b9ad-66bc1080a49a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 15:46:27.890262 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:46:27.890262 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:46:27.890262 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:46:27.890262 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:46:27.890262 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:46:27.890262 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:46:27.890262 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:46:27.890262 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:46:27.890262 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:46:27.890262 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:46:27.890262 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:46:27.890262 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 15:46:27.890262 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 15:46:27.890262 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 15:46:27.890262 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 15:46:27.891174 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:46:27.891174 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:46:27.891174 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:46:27.891174 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:46:27.891174 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:46:27.891174 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:46:27.891174 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:46:27.891174 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Aug 30 15:46:27.891174 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:46:27.891174 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:46:27.891174 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:46:27.891174 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:46:27.891174 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:46:27.891174 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:46:27.891174 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:46:27.891174 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:46:27.891174 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 15:46:27.894244 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 15:46:27.894244 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 15:46:27.894244 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Aug 30 15:46:27.894244 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 229/463] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 30 15:46:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 15:46:27.894244 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3abdb12-829f-43c6-b07b-080e6ca68285 None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:27.894244 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3abdb12-829f-43c6-b07b-080e6ca68285 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:27.894244 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3abdb12-829f-43c6-b07b-080e6ca68285 None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:27.894900 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 230/464] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 30 15:46:27 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 15:46:27.899167 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3abdb12-829f-43c6-b07b-080e6ca68285 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:27.901963 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bee3a3-653d-456d-8c1a-ed945ac449a5 None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:27.937024 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5baf02e-5390-46a5-baf2-0099873f9a61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:27.937792 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5baf02e-5390-46a5-baf2-0099873f9a61 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:27.938076 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5baf02e-5390-46a5-baf2-0099873f9a61 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:27.938347 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5baf02e-5390-46a5-baf2-0099873f9a61 None admin] PATH_INFO: `/v3/domains` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:27.939019 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5baf02e-5390-46a5-baf2-0099873f9a61 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81512) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:46:27.939323 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5baf02e-5390-46a5-baf2-0099873f9a61 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:27.940723 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5baf02e-5390-46a5-baf2-0099873f9a61 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:27.957264 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 235/465] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Aug 30 15:46:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:46:27.957264 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5baf02e-5390-46a5-baf2-0099873f9a61 None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:28.008527 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3abdb12-829f-43c6-b07b-080e6ca68285 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:28.015280 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:28.017857 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:28.047113 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0ada2e-cb64-457f-8f28-e1facaf64924 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:28.047739 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0ada2e-cb64-457f-8f28-e1facaf64924 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:28.048012 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0ada2e-cb64-457f-8f28-e1facaf64924 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:28.048275 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0ada2e-cb64-457f-8f28-e1facaf64924 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:28.058696 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0ada2e-cb64-457f-8f28-e1facaf64924 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:28.058908 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5baf02e-5390-46a5-baf2-0099873f9a61 None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:28.060163 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0ada2e-cb64-457f-8f28-e1facaf64924 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:28.074829 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-dc0ada2e-cb64-457f-8f28-e1facaf64924 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 15:46:28.074829 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:46:28.074829 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:46:28.074829 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:46:28.074829 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:46:28.074829 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:46:28.074829 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:46:28.074829 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:46:28.074829 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:46:28.074829 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:46:28.074829 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:46:28.074829 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:46:28.074829 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 15:46:28.074829 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 15:46:28.074829 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 15:46:28.074829 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 15:46:28.075768 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:46:28.075768 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:46:28.075768 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:46:28.075768 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:46:28.075768 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:46:28.075768 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:46:28.075768 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:46:28.075768 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Aug 30 15:46:28.075768 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:46:28.075768 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:46:28.075768 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:46:28.075768 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:46:28.075768 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:46:28.075768 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:46:28.075768 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:46:28.075768 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:46:28.075768 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 15:46:28.076603 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 15:46:28.076603 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 15:46:28.076603 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Aug 30 15:46:28.080957 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 231/466] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 30 15:46:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 15:46:28.082248 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fdf4cf-d39d-4a13-9aba-d5ee04067431 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:28.082523 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fdf4cf-d39d-4a13-9aba-d5ee04067431 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:28.082791 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fdf4cf-d39d-4a13-9aba-d5ee04067431 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:28.096680 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12f71f2-5a69-4289-87bf-281669f4ba97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:28.097323 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12f71f2-5a69-4289-87bf-281669f4ba97 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:28.097606 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12f71f2-5a69-4289-87bf-281669f4ba97 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:28.097875 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12f71f2-5a69-4289-87bf-281669f4ba97 None admin] PATH_INFO: `/v3/domains` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:28.098608 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12f71f2-5a69-4289-87bf-281669f4ba97 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81512) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:46:28.099016 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12f71f2-5a69-4289-87bf-281669f4ba97 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:28.100328 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-40fdf4cf-d39d-4a13-9aba-d5ee04067431 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:46:28.101155 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12f71f2-5a69-4289-87bf-281669f4ba97 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:28.102387 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-40fdf4cf-d39d-4a13-9aba-d5ee04067431 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:46:28.119573 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 236/467] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Aug 30 15:46:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:46:28.119573 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12f71f2-5a69-4289-87bf-281669f4ba97 None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:28.178268 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 232/468] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 30 15:46:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 15:46:28.179131 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fdf4cf-d39d-4a13-9aba-d5ee04067431 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:28.239455 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12f71f2-5a69-4289-87bf-281669f4ba97 None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:28.259356 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:28.273313 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:28.284278 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d899b670-8130-4848-b22c-40f17ac4787d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:28.285441 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d899b670-8130-4848-b22c-40f17ac4787d None admin] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:28.285828 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d899b670-8130-4848-b22c-40f17ac4787d None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:28.286202 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d899b670-8130-4848-b22c-40f17ac4787d None admin] PATH_INFO: `/v3/users` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:28.287168 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d899b670-8130-4848-b22c-40f17ac4787d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:28.288574 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fdf4cf-d39d-4a13-9aba-d5ee04067431 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:28.288809 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d899b670-8130-4848-b22c-40f17ac4787d None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:28.321247 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 237/469] 10.222.0.20 () {60 vars in 1132 bytes} [Tue Aug 30 15:46:28 2022] POST /identity/v3/users => generated 246 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 15:46:28.321247 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d899b670-8130-4848-b22c-40f17ac4787d None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:28.345229 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd87643-bbdf-4bd1-876f-f68c3cfe3cda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:28.347339 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd87643-bbdf-4bd1-876f-f68c3cfe3cda None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:28.347614 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd87643-bbdf-4bd1-876f-f68c3cfe3cda None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:28.350087 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd87643-bbdf-4bd1-876f-f68c3cfe3cda None admin] PATH_INFO: `/v3/domains` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:28.352577 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd87643-bbdf-4bd1-876f-f68c3cfe3cda None admin] RBAC: Adding query filter params (name=Default) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:46:28.352903 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd87643-bbdf-4bd1-876f-f68c3cfe3cda None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:28.355703 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd87643-bbdf-4bd1-876f-f68c3cfe3cda None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:28.384671 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 233/470] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Aug 30 15:46:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:46:28.390599 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd87643-bbdf-4bd1-876f-f68c3cfe3cda None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:28.525739 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d899b670-8130-4848-b22c-40f17ac4787d None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:28.534512 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd87643-bbdf-4bd1-876f-f68c3cfe3cda None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:28.563890 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebd9f7a-4845-4b01-bc0a-c1eccb00c476 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:28.564642 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebd9f7a-4845-4b01-bc0a-c1eccb00c476 None admin] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:28.564642 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebd9f7a-4845-4b01-bc0a-c1eccb00c476 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:28.565255 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebd9f7a-4845-4b01-bc0a-c1eccb00c476 None admin] PATH_INFO: `/v3/users` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:28.566207 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebd9f7a-4845-4b01-bc0a-c1eccb00c476 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:28.568128 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebd9f7a-4845-4b01-bc0a-c1eccb00c476 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:28.572789 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb7c53d-058b-4ee0-9b89-ee249ae32d80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:28.573462 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb7c53d-058b-4ee0-9b89-ee249ae32d80 None admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:28.573739 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb7c53d-058b-4ee0-9b89-ee249ae32d80 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:28.574007 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb7c53d-058b-4ee0-9b89-ee249ae32d80 None admin] PATH_INFO: `/v3/users` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:28.574657 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb7c53d-058b-4ee0-9b89-ee249ae32d80 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:28.576091 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb7c53d-058b-4ee0-9b89-ee249ae32d80 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:28.594566 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 238/471] 10.222.0.20 () {60 vars in 1132 bytes} [Tue Aug 30 15:46:28 2022] POST /identity/v3/users => generated 248 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 15:46:28.595421 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebd9f7a-4845-4b01-bc0a-c1eccb00c476 None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:28.616199 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 234/472] 10.222.0.20 () {60 vars in 1132 bytes} [Tue Aug 30 15:46:28 2022] POST /identity/v3/users => generated 248 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 15:46:28.649720 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:28.686922 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72999103-42bc-4889-9252-3daf6b21d8e0 None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:28.687204 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72999103-42bc-4889-9252-3daf6b21d8e0 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:28.687480 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72999103-42bc-4889-9252-3daf6b21d8e0 None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:28.689039 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 235/473] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 30 15:46:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 15:46:28.694858 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52672509-cc3d-4aae-98d3-a8f601d1b0a0 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:28.695122 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52672509-cc3d-4aae-98d3-a8f601d1b0a0 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:28.695400 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52672509-cc3d-4aae-98d3-a8f601d1b0a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:28.718733 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:28.741447 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-52672509-cc3d-4aae-98d3-a8f601d1b0a0 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:46:28.742685 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-52672509-cc3d-4aae-98d3-a8f601d1b0a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:46:28.756508 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebd9f7a-4845-4b01-bc0a-c1eccb00c476 None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:28.822770 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37e4151-b796-4150-bd1a-017a0459f704 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:28.823329 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37e4151-b796-4150-bd1a-017a0459f704 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:28.823605 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37e4151-b796-4150-bd1a-017a0459f704 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:28.823870 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37e4151-b796-4150-bd1a-017a0459f704 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:28.837365 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37e4151-b796-4150-bd1a-017a0459f704 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:28.839051 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37e4151-b796-4150-bd1a-017a0459f704 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:28.858175 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application [None req-a37e4151-b796-4150-bd1a-017a0459f704 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 15:46:28.858175 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:46:28.858175 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:46:28.858175 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:46:28.858175 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:46:28.858175 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:46:28.858175 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:46:28.858175 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:46:28.858175 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:46:28.858175 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:46:28.858175 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:46:28.858175 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:46:28.858175 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 15:46:28.858175 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 15:46:28.858175 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 15:46:28.858175 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 15:46:28.859206 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:46:28.859206 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:46:28.859206 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:46:28.859206 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:46:28.859206 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:46:28.859206 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:46:28.859206 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:46:28.859206 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application with Lock( Aug 30 15:46:28.859206 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:46:28.859206 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:46:28.859206 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:46:28.859206 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:46:28.859206 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:46:28.859206 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:46:28.859206 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:46:28.859206 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:46:28.859206 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 15:46:28.860043 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 15:46:28.860043 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 15:46:28.860043 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Aug 30 15:46:28.860043 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 239/474] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 30 15:46:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 15:46:28.865273 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37e4151-b796-4150-bd1a-017a0459f704 None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:28.866265 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 236/475] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 30 15:46:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 15:46:28.952524 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:28.978031 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37e4151-b796-4150-bd1a-017a0459f704 None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:29.017049 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d0de0b-9690-4c8e-a04c-54c2b1b39b39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:29.017712 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d0de0b-9690-4c8e-a04c-54c2b1b39b39 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:29.017991 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d0de0b-9690-4c8e-a04c-54c2b1b39b39 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:29.018263 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d0de0b-9690-4c8e-a04c-54c2b1b39b39 None admin] PATH_INFO: `/v3/domains` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:29.018922 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d0de0b-9690-4c8e-a04c-54c2b1b39b39 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81512) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:46:29.019282 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d0de0b-9690-4c8e-a04c-54c2b1b39b39 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:29.020699 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d0de0b-9690-4c8e-a04c-54c2b1b39b39 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:29.033666 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 240/476] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Aug 30 15:46:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 15:46:29.039118 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52672509-cc3d-4aae-98d3-a8f601d1b0a0 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:29.146559 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52672509-cc3d-4aae-98d3-a8f601d1b0a0 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:29.182345 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee8cc1d-ba7c-41d1-83ba-38705e4b259b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:29.183254 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee8cc1d-ba7c-41d1-83ba-38705e4b259b None admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:29.183704 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee8cc1d-ba7c-41d1-83ba-38705e4b259b None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:29.184141 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee8cc1d-ba7c-41d1-83ba-38705e4b259b None admin] PATH_INFO: `/v3/users` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:29.184981 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee8cc1d-ba7c-41d1-83ba-38705e4b259b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:29.186679 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee8cc1d-ba7c-41d1-83ba-38705e4b259b None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:29.196582 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:29.202340 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d0de0b-9690-4c8e-a04c-54c2b1b39b39 None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:29.213271 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 237/477] 10.222.0.20 () {60 vars in 1132 bytes} [Tue Aug 30 15:46:29 2022] POST /identity/v3/users => generated 249 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 15:46:29.355903 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d0de0b-9690-4c8e-a04c-54c2b1b39b39 None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:29.393260 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d3ecc9-095e-4c05-bd88-77bd4b0343c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:29.394265 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d3ecc9-095e-4c05-bd88-77bd4b0343c1 None admin] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:29.394751 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d3ecc9-095e-4c05-bd88-77bd4b0343c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:29.395216 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d3ecc9-095e-4c05-bd88-77bd4b0343c1 None admin] PATH_INFO: `/v3/roles` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:29.396099 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d3ecc9-095e-4c05-bd88-77bd4b0343c1 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:29.397799 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d3ecc9-095e-4c05-bd88-77bd4b0343c1 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:29.408668 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.sql.core [None req-31d3ecc9-095e-4c05-bd88-77bd4b0343c1 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 30 15:46:29.408668 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 30 15:46:29.408668 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [parameters: {'id': 'c21eba5118714e51a66e824c29a4f5b9', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 30 15:46:29.408668 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81512) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 30 15:46:29.427717 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application [None req-31d3ecc9-095e-4c05-bd88-77bd4b0343c1 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 30 15:46:29.427717 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:46:29.427717 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 30 15:46:29.427717 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 30 15:46:29.427717 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 30 15:46:29.427717 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 30 15:46:29.427717 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 30 15:46:29.427717 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application result = self._query(query) Aug 30 15:46:29.427717 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 30 15:46:29.427717 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application conn.query(q) Aug 30 15:46:29.427717 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 30 15:46:29.427717 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 30 15:46:29.427717 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 30 15:46:29.427717 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application result.read() Aug 30 15:46:29.427717 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 30 15:46:29.429134 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 30 15:46:29.429134 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 30 15:46:29.429134 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 30 15:46:29.429134 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 30 15:46:29.429134 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 30 15:46:29.429134 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 30 15:46:29.429134 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 30 15:46:29.429134 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 30 15:46:29.429134 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Aug 30 15:46:29.429134 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 30 15:46:29.429134 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Aug 30 15:46:29.429134 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:46:29.429134 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 30 15:46:29.429134 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 30 15:46:29.429134 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Aug 30 15:46:29.429134 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return ref.to_dict() Aug 30 15:46:29.429134 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 30 15:46:29.429134 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application next(self.gen) Aug 30 15:46:29.430152 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 30 15:46:29.430152 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application yield resource Aug 30 15:46:29.430152 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 30 15:46:29.430152 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application next(self.gen) Aug 30 15:46:29.430152 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 30 15:46:29.430152 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application self.session.rollback() Aug 30 15:46:29.430152 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 30 15:46:29.430152 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application self.force_reraise() Aug 30 15:46:29.430152 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 30 15:46:29.430152 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application raise self.value Aug 30 15:46:29.430152 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 30 15:46:29.430152 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 30 15:46:29.430152 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 30 15:46:29.430152 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application session.commit() Aug 30 15:46:29.430152 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 30 15:46:29.430152 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 30 15:46:29.430152 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 30 15:46:29.432592 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application self._prepare_impl() Aug 30 15:46:29.432592 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 30 15:46:29.432592 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application self.session.flush() Aug 30 15:46:29.432592 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 30 15:46:29.432592 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application self._flush(objects) Aug 30 15:46:29.432592 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 30 15:46:29.432592 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 30 15:46:29.432592 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 30 15:46:29.432592 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application compat.raise_( Aug 30 15:46:29.432592 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 30 15:46:29.432592 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application raise exception Aug 30 15:46:29.432592 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 30 15:46:29.432592 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application flush_context.execute() Aug 30 15:46:29.432592 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 30 15:46:29.432592 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application rec.execute(self) Aug 30 15:46:29.432592 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 30 15:46:29.432592 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 30 15:46:29.434680 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 30 15:46:29.434680 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 30 15:46:29.434680 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 30 15:46:29.434680 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 30 15:46:29.434680 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 30 15:46:29.434680 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 30 15:46:29.434680 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 30 15:46:29.434680 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 30 15:46:29.434680 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 30 15:46:29.434680 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 30 15:46:29.434680 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 30 15:46:29.434680 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 30 15:46:29.434680 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 30 15:46:29.434680 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 30 15:46:29.434680 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 30 15:46:29.434680 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application raise exception Aug 30 15:46:29.436705 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 30 15:46:29.436705 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 30 15:46:29.436705 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 30 15:46:29.436705 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 30 15:46:29.436705 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 30 15:46:29.436705 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application result = self._query(query) Aug 30 15:46:29.436705 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 30 15:46:29.436705 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application conn.query(q) Aug 30 15:46:29.436705 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 30 15:46:29.436705 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 30 15:46:29.436705 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 30 15:46:29.436705 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application result.read() Aug 30 15:46:29.436705 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 30 15:46:29.436705 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 30 15:46:29.436705 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 30 15:46:29.436705 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 30 15:46:29.438682 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 30 15:46:29.438682 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 30 15:46:29.438682 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 30 15:46:29.438682 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 30 15:46:29.438682 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 30 15:46:29.438682 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 30 15:46:29.438682 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application [parameters: {'id': 'c21eba5118714e51a66e824c29a4f5b9', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 30 15:46:29.438682 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 30 15:46:29.438682 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Aug 30 15:46:29.438682 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 15:46:29.438682 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Aug 30 15:46:29.438682 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:46:29.438682 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:46:29.438682 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:46:29.438682 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:46:29.438682 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:46:29.440655 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:46:29.440655 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:46:29.440655 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:46:29.440655 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:46:29.440655 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:46:29.440655 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:46:29.440655 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Aug 30 15:46:29.440655 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Aug 30 15:46:29.440655 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:46:29.440655 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:46:29.440655 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Aug 30 15:46:29.440655 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Aug 30 15:46:29.440655 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 30 15:46:29.440655 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 30 15:46:29.440655 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 30 15:46:29.440655 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Aug 30 15:46:29.444684 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 241/478] 10.222.0.20 () {60 vars in 1131 bytes} [Tue Aug 30 15:46:29 2022] POST /identity/v3/roles => generated 144 bytes in 243 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 30 15:46:29.449310 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee8cc1d-ba7c-41d1-83ba-38705e4b259b None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:29.572521 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee8cc1d-ba7c-41d1-83ba-38705e4b259b None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:29.609837 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac68986-2c5d-43eb-9d68-887a8c85bcb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:29.610482 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac68986-2c5d-43eb-9d68-887a8c85bcb6 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:29.610757 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac68986-2c5d-43eb-9d68-887a8c85bcb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:29.611021 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac68986-2c5d-43eb-9d68-887a8c85bcb6 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:29.621991 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac68986-2c5d-43eb-9d68-887a8c85bcb6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 15:46:29.628305 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-dac68986-2c5d-43eb-9d68-887a8c85bcb6 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 15:46:29.628305 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:46:29.628305 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:46:29.628305 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:46:29.628305 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:46:29.628305 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:46:29.628305 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:46:29.628305 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:46:29.628305 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:46:29.628305 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:46:29.628305 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:46:29.628305 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:46:29.628305 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 15:46:29.628305 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 15:46:29.628305 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 15:46:29.628305 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 15:46:29.629248 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 15:46:29.629248 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 15:46:29.629248 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 15:46:29.629248 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 15:46:29.629248 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:46:29.629248 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:46:29.629248 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:46:29.629248 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:46:29.629248 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:46:29.629248 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:46:29.629248 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:46:29.629248 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Aug 30 15:46:29.629248 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:46:29.629248 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:46:29.629248 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:46:29.629248 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:46:29.630070 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:46:29.630070 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:46:29.630070 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:46:29.630070 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:46:29.630070 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 15:46:29.630070 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 15:46:29.630070 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 15:46:29.630070 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 15:46:29.630070 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 15:46:29.630070 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 15:46:29.630070 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 15:46:29.630070 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Aug 30 15:46:29.630070 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 238/479] 10.222.0.20 () {56 vars in 1114 bytes} [Tue Aug 30 15:46:29 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 15:46:29.634177 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d3ecc9-095e-4c05-bd88-77bd4b0343c1 None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:29.654750 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:29.763842 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d3ecc9-095e-4c05-bd88-77bd4b0343c1 None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:29.808791 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3670cb7d-40be-447f-9a8d-3e9280acff2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:29.809429 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3670cb7d-40be-447f-9a8d-3e9280acff2b None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:29.809704 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3670cb7d-40be-447f-9a8d-3e9280acff2b None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:29.809970 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3670cb7d-40be-447f-9a8d-3e9280acff2b None admin] PATH_INFO: `/v3/roles` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:29.810638 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3670cb7d-40be-447f-9a8d-3e9280acff2b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81512) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:46:29.810935 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3670cb7d-40be-447f-9a8d-3e9280acff2b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:29.812329 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3670cb7d-40be-447f-9a8d-3e9280acff2b None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:29.824533 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 242/480] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Aug 30 15:46:29 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:46:30.629974 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62242c02-cda4-48e7-8d11-3986fa032e00 None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:30.630493 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62242c02-cda4-48e7-8d11-3986fa032e00 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:30.630493 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62242c02-cda4-48e7-8d11-3986fa032e00 None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:30.631847 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 239/481] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 30 15:46:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 15:46:30.637810 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5e883e-6366-4b6e-951b-b3a82f23ca75 None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:30.638277 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5e883e-6366-4b6e-951b-b3a82f23ca75 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:30.638372 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5e883e-6366-4b6e-951b-b3a82f23ca75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:30.665328 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-dd5e883e-6366-4b6e-951b-b3a82f23ca75 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:46:30.666526 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-dd5e883e-6366-4b6e-951b-b3a82f23ca75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:46:30.754448 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:30.760081 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 243/482] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 30 15:46:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 15:46:30.897711 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc24870-5174-4ec9-9d8b-1f4c954d0311 None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:30.897916 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc24870-5174-4ec9-9d8b-1f4c954d0311 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:30.898167 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc24870-5174-4ec9-9d8b-1f4c954d0311 None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:30.899576 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 240/483] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 30 15:46:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 15:46:30.905504 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe9f9d3-28c4-4dee-87d9-0a87e6cf2a1c None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:30.905799 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe9f9d3-28c4-4dee-87d9-0a87e6cf2a1c None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:30.906047 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe9f9d3-28c4-4dee-87d9-0a87e6cf2a1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:30.930728 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-ffe9f9d3-28c4-4dee-87d9-0a87e6cf2a1c None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:46:30.931916 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-ffe9f9d3-28c4-4dee-87d9-0a87e6cf2a1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:46:31.020871 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 244/484] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 30 15:46:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 15:46:31.190603 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d264e272-21ce-4429-aea9-8a8e2ecad335 None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:31.190792 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d264e272-21ce-4429-aea9-8a8e2ecad335 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:31.191068 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d264e272-21ce-4429-aea9-8a8e2ecad335 None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:31.192727 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 241/485] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 30 15:46:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 15:46:31.198092 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962e675b-da78-4fe6-a687-769da4820da8 None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:31.198372 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962e675b-da78-4fe6-a687-769da4820da8 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:31.198639 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962e675b-da78-4fe6-a687-769da4820da8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:31.221248 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-962e675b-da78-4fe6-a687-769da4820da8 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:46:31.222422 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-962e675b-da78-4fe6-a687-769da4820da8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:46:31.230245 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d264e272-21ce-4429-aea9-8a8e2ecad335 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:31.316754 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:31.351438 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 245/486] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 30 15:46:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 15:46:31.372365 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962e675b-da78-4fe6-a687-769da4820da8 None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:31.394165 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d264e272-21ce-4429-aea9-8a8e2ecad335 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:31.428492 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9ca9ed-20a8-4091-a788-6519d68312a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:31.429130 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9ca9ed-20a8-4091-a788-6519d68312a2 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:31.429437 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9ca9ed-20a8-4091-a788-6519d68312a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:31.429709 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9ca9ed-20a8-4091-a788-6519d68312a2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:31.442969 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9ca9ed-20a8-4091-a788-6519d68312a2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 15:46:31.449115 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-7b9ca9ed-20a8-4091-a788-6519d68312a2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 15:46:31.449115 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:46:31.449115 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:46:31.449115 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:46:31.449115 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:46:31.449115 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:46:31.449115 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:46:31.449115 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:46:31.449115 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:46:31.449115 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:46:31.449115 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:46:31.449115 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:46:31.449115 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 15:46:31.449115 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 15:46:31.449115 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 15:46:31.449115 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 15:46:31.450132 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 15:46:31.450132 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 15:46:31.450132 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 15:46:31.450132 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 15:46:31.450132 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:46:31.450132 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:46:31.450132 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:46:31.450132 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:46:31.450132 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:46:31.450132 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:46:31.450132 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:46:31.450132 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Aug 30 15:46:31.450132 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:46:31.450132 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:46:31.450132 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:46:31.450132 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:46:31.450925 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:46:31.450925 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:46:31.450925 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:46:31.450925 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:46:31.450925 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 15:46:31.450925 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 15:46:31.450925 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 15:46:31.450925 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 15:46:31.450925 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 15:46:31.450925 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 15:46:31.450925 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 15:46:31.450925 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Aug 30 15:46:31.450925 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 242/487] 10.222.0.20 () {56 vars in 1102 bytes} [Tue Aug 30 15:46:31 2022] GET /identity/v3/roles/service => generated 85 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 15:46:31.457538 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9ca9ed-20a8-4091-a788-6519d68312a2 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:31.493245 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:31.535235 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962e675b-da78-4fe6-a687-769da4820da8 None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:31.607699 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f83d904-f052-4828-b9da-0d957d90911e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:31.608350 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f83d904-f052-4828-b9da-0d957d90911e None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:31.608657 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f83d904-f052-4828-b9da-0d957d90911e None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:31.608927 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f83d904-f052-4828-b9da-0d957d90911e None admin] PATH_INFO: `/v3/roles/service` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:31.637256 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f83d904-f052-4828-b9da-0d957d90911e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 15:46:31.640467 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application [None req-5f83d904-f052-4828-b9da-0d957d90911e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 15:46:31.640467 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:46:31.640467 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:46:31.640467 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:46:31.640467 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:46:31.640467 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:46:31.640467 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:46:31.640467 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:46:31.640467 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:46:31.640467 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:46:31.640467 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:46:31.640467 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:46:31.640467 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 15:46:31.640467 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 15:46:31.640467 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 15:46:31.640467 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 15:46:31.641805 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 15:46:31.641805 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 15:46:31.641805 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 15:46:31.641805 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 15:46:31.641805 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:46:31.641805 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:46:31.641805 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:46:31.641805 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:46:31.641805 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:46:31.641805 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:46:31.641805 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:46:31.641805 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application with Lock( Aug 30 15:46:31.641805 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:46:31.641805 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:46:31.641805 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:46:31.641805 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:46:31.642613 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:46:31.642613 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:46:31.642613 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:46:31.642613 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:46:31.642613 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 15:46:31.642613 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 15:46:31.642613 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 15:46:31.642613 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 15:46:31.642613 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 15:46:31.642613 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 15:46:31.642613 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 15:46:31.642613 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Aug 30 15:46:31.646525 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9ca9ed-20a8-4091-a788-6519d68312a2 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:31.649844 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 246/488] 10.222.0.20 () {56 vars in 1102 bytes} [Tue Aug 30 15:46:31 2022] GET /identity/v3/roles/service => generated 85 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 15:46:31.650544 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f83d904-f052-4828-b9da-0d957d90911e None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:31.665247 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:31.689286 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc46718-a0f3-4983-bbc4-c9e1260b1f0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:31.690300 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc46718-a0f3-4983-bbc4-c9e1260b1f0b None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:31.690716 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc46718-a0f3-4983-bbc4-c9e1260b1f0b None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:31.691119 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc46718-a0f3-4983-bbc4-c9e1260b1f0b None admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:31.691801 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc46718-a0f3-4983-bbc4-c9e1260b1f0b None admin] RBAC: Adding query filter params (name=service) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:46:31.692101 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc46718-a0f3-4983-bbc4-c9e1260b1f0b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:31.694740 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc46718-a0f3-4983-bbc4-c9e1260b1f0b None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:31.705349 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 243/489] 10.222.0.20 () {56 vars in 1103 bytes} [Tue Aug 30 15:46:31 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:46:31.707076 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc46718-a0f3-4983-bbc4-c9e1260b1f0b None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:31.774822 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f83d904-f052-4828-b9da-0d957d90911e None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:31.797273 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:31.816815 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc46718-a0f3-4983-bbc4-c9e1260b1f0b None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:31.820126 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adebd979-34a0-40b4-82bf-b38b76f7dd9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:31.820957 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adebd979-34a0-40b4-82bf-b38b76f7dd9c None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:31.821409 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adebd979-34a0-40b4-82bf-b38b76f7dd9c None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:31.821845 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adebd979-34a0-40b4-82bf-b38b76f7dd9c None admin] PATH_INFO: `/v3/roles/service` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:31.834920 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adebd979-34a0-40b4-82bf-b38b76f7dd9c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 15:46:31.841320 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application [None req-adebd979-34a0-40b4-82bf-b38b76f7dd9c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 15:46:31.841320 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:46:31.841320 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:46:31.841320 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:46:31.841320 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:46:31.841320 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:46:31.841320 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:46:31.841320 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:46:31.841320 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:46:31.841320 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:46:31.841320 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:46:31.841320 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:46:31.841320 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 15:46:31.841320 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 15:46:31.841320 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 15:46:31.841320 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 15:46:31.842405 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 15:46:31.842405 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 15:46:31.842405 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 15:46:31.842405 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 15:46:31.842405 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:46:31.842405 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:46:31.842405 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:46:31.842405 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:46:31.842405 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:46:31.842405 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:46:31.842405 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:46:31.842405 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application with Lock( Aug 30 15:46:31.842405 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:46:31.842405 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:46:31.842405 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:46:31.842405 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:46:31.843440 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:46:31.843440 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:46:31.843440 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:46:31.843440 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:46:31.843440 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 15:46:31.843440 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 15:46:31.843440 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 15:46:31.843440 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 15:46:31.843440 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 15:46:31.843440 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 15:46:31.843440 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 15:46:31.843440 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Aug 30 15:46:31.846363 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 247/490] 10.222.0.20 () {56 vars in 1102 bytes} [Tue Aug 30 15:46:31 2022] GET /identity/v3/roles/service => generated 85 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 15:46:31.847902 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adebd979-34a0-40b4-82bf-b38b76f7dd9c None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:31.855792 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3954cb-5d2b-49f9-ba7e-c1526745ef3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:31.856496 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3954cb-5d2b-49f9-ba7e-c1526745ef3a None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:31.856840 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3954cb-5d2b-49f9-ba7e-c1526745ef3a None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:31.857178 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3954cb-5d2b-49f9-ba7e-c1526745ef3a None admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:31.857955 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3954cb-5d2b-49f9-ba7e-c1526745ef3a None admin] RBAC: Adding query filter params (name=service) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:46:31.858325 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3954cb-5d2b-49f9-ba7e-c1526745ef3a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:31.859842 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3954cb-5d2b-49f9-ba7e-c1526745ef3a None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:31.873317 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 244/491] 10.222.0.20 () {56 vars in 1103 bytes} [Tue Aug 30 15:46:31 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:46:31.874390 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3954cb-5d2b-49f9-ba7e-c1526745ef3a None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:31.910718 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:31.972546 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adebd979-34a0-40b4-82bf-b38b76f7dd9c None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:31.982075 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3954cb-5d2b-49f9-ba7e-c1526745ef3a None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:32.008467 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e9bf75-df31-43a8-8d6a-062cdb135e45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:32.008913 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e9bf75-df31-43a8-8d6a-062cdb135e45 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:32.009185 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e9bf75-df31-43a8-8d6a-062cdb135e45 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:32.009519 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e9bf75-df31-43a8-8d6a-062cdb135e45 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:32.016267 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031f0b23-d32f-48f4-b7a8-f664e0a8e2e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:32.016874 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031f0b23-d32f-48f4-b7a8-f664e0a8e2e3 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:32.017151 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031f0b23-d32f-48f4-b7a8-f664e0a8e2e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:32.017466 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031f0b23-d32f-48f4-b7a8-f664e0a8e2e3 None admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:32.018134 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031f0b23-d32f-48f4-b7a8-f664e0a8e2e3 None admin] RBAC: Adding query filter params (name=service) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:46:32.018442 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031f0b23-d32f-48f4-b7a8-f664e0a8e2e3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:32.019828 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031f0b23-d32f-48f4-b7a8-f664e0a8e2e3 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:32.020026 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e9bf75-df31-43a8-8d6a-062cdb135e45 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:32.021590 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e9bf75-df31-43a8-8d6a-062cdb135e45 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:32.028407 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 245/492] 10.222.0.20 () {56 vars in 1103 bytes} [Tue Aug 30 15:46:31 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:46:32.029026 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031f0b23-d32f-48f4-b7a8-f664e0a8e2e3 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:32.041370 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application [None req-81e9bf75-df31-43a8-8d6a-062cdb135e45 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 15:46:32.041370 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:46:32.041370 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:46:32.041370 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:46:32.041370 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:46:32.041370 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:46:32.041370 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:46:32.041370 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:46:32.041370 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:46:32.041370 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:46:32.041370 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:46:32.041370 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:46:32.041370 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 15:46:32.041370 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 15:46:32.041370 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 15:46:32.041370 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 15:46:32.042336 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:46:32.042336 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:46:32.042336 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:46:32.042336 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:46:32.042336 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:46:32.042336 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:46:32.042336 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:46:32.042336 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application with Lock( Aug 30 15:46:32.042336 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:46:32.042336 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:46:32.042336 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:46:32.042336 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:46:32.042336 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:46:32.042336 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:46:32.042336 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:46:32.042336 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:46:32.042336 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 15:46:32.043184 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 15:46:32.043184 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 15:46:32.043184 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Aug 30 15:46:32.043184 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 248/493] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 30 15:46:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 15:46:32.044157 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13e004c-b390-43bf-aec2-f9ea82449965 None None] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:32.044157 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13e004c-b390-43bf-aec2-f9ea82449965 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:32.044157 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13e004c-b390-43bf-aec2-f9ea82449965 None None] PATH_INFO: `/` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:32.045468 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 249/494] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 30 15:46:32 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 15:46:32.050075 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13e004c-b390-43bf-aec2-f9ea82449965 None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:32.082314 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:32.132039 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:32.139653 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031f0b23-d32f-48f4-b7a8-f664e0a8e2e3 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:32.165002 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13e004c-b390-43bf-aec2-f9ea82449965 None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:32.176158 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab4afd2-feb6-4f4e-b02b-3de6cf3b33a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:32.177287 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab4afd2-feb6-4f4e-b02b-3de6cf3b33a8 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:32.177785 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab4afd2-feb6-4f4e-b02b-3de6cf3b33a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:32.178225 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab4afd2-feb6-4f4e-b02b-3de6cf3b33a8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:32.190425 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab4afd2-feb6-4f4e-b02b-3de6cf3b33a8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:32.191897 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab4afd2-feb6-4f4e-b02b-3de6cf3b33a8 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:32.199344 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99591459-8090-42eb-940d-75d5744398ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:32.199972 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99591459-8090-42eb-940d-75d5744398ce None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:32.200249 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99591459-8090-42eb-940d-75d5744398ce None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:32.200514 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99591459-8090-42eb-940d-75d5744398ce None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:32.207256 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-eab4afd2-feb6-4f4e-b02b-3de6cf3b33a8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 15:46:32.207256 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:46:32.207256 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:46:32.207256 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:46:32.207256 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:46:32.207256 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:46:32.207256 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:46:32.207256 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:46:32.207256 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:46:32.207256 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:46:32.207256 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:46:32.207256 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:46:32.207256 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 15:46:32.207256 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 15:46:32.207256 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 15:46:32.207256 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 15:46:32.208288 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:46:32.208288 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:46:32.208288 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:46:32.208288 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:46:32.208288 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:46:32.208288 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:46:32.208288 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:46:32.208288 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Aug 30 15:46:32.208288 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:46:32.208288 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:46:32.208288 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:46:32.208288 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:46:32.208288 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:46:32.208288 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:46:32.208288 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:46:32.208288 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:46:32.208288 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 15:46:32.209251 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 15:46:32.209251 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 15:46:32.209251 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Aug 30 15:46:32.211031 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 246/495] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 30 15:46:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 15:46:32.212300 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab4afd2-feb6-4f4e-b02b-3de6cf3b33a8 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:32.218729 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99591459-8090-42eb-940d-75d5744398ce None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:32.220354 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99591459-8090-42eb-940d-75d5744398ce None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:32.236115 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application [None req-99591459-8090-42eb-940d-75d5744398ce None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 15:46:32.236115 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:46:32.236115 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:46:32.236115 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:46:32.236115 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:46:32.236115 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:46:32.236115 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:46:32.236115 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:46:32.236115 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:46:32.236115 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:46:32.236115 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:46:32.236115 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:46:32.236115 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 15:46:32.236115 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 15:46:32.236115 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 15:46:32.236115 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 15:46:32.237029 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:46:32.237029 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:46:32.237029 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:46:32.237029 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:46:32.237029 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:46:32.237029 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:46:32.237029 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:46:32.237029 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application with Lock( Aug 30 15:46:32.237029 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:46:32.237029 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:46:32.237029 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:46:32.237029 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:46:32.237029 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:46:32.237029 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:46:32.237029 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:46:32.237029 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:46:32.237029 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 15:46:32.238422 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 15:46:32.238422 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 15:46:32.238422 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Aug 30 15:46:32.238422 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 250/496] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 30 15:46:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 15:46:32.243269 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8c1b5a-d719-474b-833d-736fe7808ad7 None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:32.243546 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8c1b5a-d719-474b-833d-736fe7808ad7 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:32.243809 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8c1b5a-d719-474b-833d-736fe7808ad7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:32.262582 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-3d8c1b5a-d719-474b-833d-736fe7808ad7 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:46:32.263768 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-3d8c1b5a-d719-474b-833d-736fe7808ad7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:46:32.320625 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab4afd2-feb6-4f4e-b02b-3de6cf3b33a8 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:32.334869 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 251/497] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 30 15:46:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 15:46:32.336264 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8c1b5a-d719-474b-833d-736fe7808ad7 None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:32.353706 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:32.356824 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74579319-bee7-4586-b906-828ab131b7b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:32.357492 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74579319-bee7-4586-b906-828ab131b7b7 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:32.357772 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74579319-bee7-4586-b906-828ab131b7b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:32.358037 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74579319-bee7-4586-b906-828ab131b7b7 None admin] PATH_INFO: `/v3/domains` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:32.358726 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74579319-bee7-4586-b906-828ab131b7b7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:46:32.359028 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74579319-bee7-4586-b906-828ab131b7b7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:32.360407 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74579319-bee7-4586-b906-828ab131b7b7 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:32.372582 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 247/498] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Aug 30 15:46:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:46:32.373405 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74579319-bee7-4586-b906-828ab131b7b7 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:32.383677 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:32.454320 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8c1b5a-d719-474b-833d-736fe7808ad7 None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:32.475131 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74579319-bee7-4586-b906-828ab131b7b7 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:32.491198 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca395aee-4adf-4ba9-bf5d-ecb78f293858 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:32.491745 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca395aee-4adf-4ba9-bf5d-ecb78f293858 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:32.492025 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca395aee-4adf-4ba9-bf5d-ecb78f293858 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:32.492298 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca395aee-4adf-4ba9-bf5d-ecb78f293858 None admin] PATH_INFO: `/v3/domains` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:32.492940 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca395aee-4adf-4ba9-bf5d-ecb78f293858 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81512) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:46:32.493259 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca395aee-4adf-4ba9-bf5d-ecb78f293858 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:32.494644 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca395aee-4adf-4ba9-bf5d-ecb78f293858 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:32.506700 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 252/499] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Aug 30 15:46:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:46:32.510765 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3300f25-f06f-4590-b602-63ec48c69732 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:32.511462 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3300f25-f06f-4590-b602-63ec48c69732 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:32.511738 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3300f25-f06f-4590-b602-63ec48c69732 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:32.512007 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3300f25-f06f-4590-b602-63ec48c69732 None admin] PATH_INFO: `/v3/domains` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:32.512299 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eea5b2f-363e-4e35-9e15-5b202c965acf None None] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:32.512585 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eea5b2f-363e-4e35-9e15-5b202c965acf None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:32.512677 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3300f25-f06f-4590-b602-63ec48c69732 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:46:32.512849 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eea5b2f-363e-4e35-9e15-5b202c965acf None None] PATH_INFO: `/` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:32.512940 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3300f25-f06f-4590-b602-63ec48c69732 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:32.514337 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 253/500] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 30 15:46:32 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 15:46:32.514944 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eea5b2f-363e-4e35-9e15-5b202c965acf None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:32.519127 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3300f25-f06f-4590-b602-63ec48c69732 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:32.531127 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 248/501] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Aug 30 15:46:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:46:32.532157 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3300f25-f06f-4590-b602-63ec48c69732 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:32.595191 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:32.605142 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:32.625198 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eea5b2f-363e-4e35-9e15-5b202c965acf None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:32.641014 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3300f25-f06f-4590-b602-63ec48c69732 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:32.660652 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6e842b-2cce-43b4-bb29-07340b121c7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:32.661309 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6e842b-2cce-43b4-bb29-07340b121c7e None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:32.661591 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6e842b-2cce-43b4-bb29-07340b121c7e None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:32.661861 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6e842b-2cce-43b4-bb29-07340b121c7e None admin] PATH_INFO: `/v3/users/nova` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:32.669447 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6e842b-2cce-43b4-bb29-07340b121c7e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:32.670931 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6e842b-2cce-43b4-bb29-07340b121c7e None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:32.675945 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99c5813-ba20-414a-9364-de8a5abcefe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:32.676570 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99c5813-ba20-414a-9364-de8a5abcefe0 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:32.676986 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99c5813-ba20-414a-9364-de8a5abcefe0 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:32.677283 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99c5813-ba20-414a-9364-de8a5abcefe0 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:32.683350 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application [None req-8e6e842b-2cce-43b4-bb29-07340b121c7e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 30 15:46:32.683350 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:46:32.683350 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:46:32.683350 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:46:32.683350 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:46:32.683350 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:46:32.683350 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:46:32.683350 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:46:32.683350 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:46:32.683350 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:46:32.683350 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:46:32.683350 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:46:32.683350 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 15:46:32.683350 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 15:46:32.683350 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 15:46:32.683350 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 15:46:32.684379 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:46:32.684379 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:46:32.684379 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 15:46:32.684379 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 15:46:32.684379 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 15:46:32.684379 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 15:46:32.684379 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:46:32.684379 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:46:32.684379 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:46:32.684379 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:46:32.684379 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:46:32.684379 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application with Lock( Aug 30 15:46:32.684379 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:46:32.684379 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:46:32.684379 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:46:32.684379 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:46:32.684379 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:46:32.684379 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:46:32.690264 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:46:32.690264 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:46:32.690264 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 15:46:32.690264 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 15:46:32.690264 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 15:46:32.690264 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 15:46:32.690264 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 15:46:32.690264 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 15:46:32.690264 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 30 15:46:32.690264 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Aug 30 15:46:32.690264 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 254/502] 10.222.0.20 () {56 vars in 1093 bytes} [Tue Aug 30 15:46:32 2022] GET /identity/v3/users/nova => generated 82 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 15:46:32.690961 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99c5813-ba20-414a-9364-de8a5abcefe0 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:32.691053 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4285cb-d0fa-4572-b81b-17a40d95e655 None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:32.691053 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4285cb-d0fa-4572-b81b-17a40d95e655 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:32.691194 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99c5813-ba20-414a-9364-de8a5abcefe0 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:32.691271 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4285cb-d0fa-4572-b81b-17a40d95e655 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:32.703792 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-d99c5813-ba20-414a-9364-de8a5abcefe0 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 30 15:46:32.703792 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:46:32.703792 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:46:32.703792 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:46:32.703792 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:46:32.703792 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:46:32.703792 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:46:32.703792 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:46:32.703792 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:46:32.703792 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:46:32.703792 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:46:32.703792 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:46:32.703792 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 15:46:32.703792 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 15:46:32.703792 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 15:46:32.703792 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 15:46:32.704898 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:46:32.704898 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:46:32.704898 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 15:46:32.704898 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 15:46:32.704898 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 15:46:32.704898 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 15:46:32.704898 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:46:32.704898 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:46:32.704898 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:46:32.704898 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:46:32.704898 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:46:32.704898 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Aug 30 15:46:32.704898 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:46:32.704898 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:46:32.704898 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:46:32.704898 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:46:32.704898 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:46:32.704898 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:46:32.706045 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:46:32.706045 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:46:32.706045 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 15:46:32.706045 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 15:46:32.706045 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 15:46:32.706045 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 15:46:32.706045 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 15:46:32.706045 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 15:46:32.706045 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 30 15:46:32.706045 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Aug 30 15:46:32.706693 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-2d4285cb-d0fa-4572-b81b-17a40d95e655 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:46:32.706774 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 249/503] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Aug 30 15:46:32 2022] GET /identity/v3/users/cinder => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 15:46:32.706852 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-2d4285cb-d0fa-4572-b81b-17a40d95e655 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:46:32.706939 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99c5813-ba20-414a-9364-de8a5abcefe0 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:32.776629 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 255/504] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 30 15:46:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 15:46:32.777440 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4285cb-d0fa-4572-b81b-17a40d95e655 None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:32.812411 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99c5813-ba20-414a-9364-de8a5abcefe0 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:32.838992 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:32.848140 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:32.848287 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be8c23d-128f-4f8f-ab74-19ed04d48135 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:32.848892 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be8c23d-128f-4f8f-ab74-19ed04d48135 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:32.849165 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be8c23d-128f-4f8f-ab74-19ed04d48135 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:32.849463 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be8c23d-128f-4f8f-ab74-19ed04d48135 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:32.857100 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be8c23d-128f-4f8f-ab74-19ed04d48135 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:32.858920 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be8c23d-128f-4f8f-ab74-19ed04d48135 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:32.869963 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-1be8c23d-128f-4f8f-ab74-19ed04d48135 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 30 15:46:32.869963 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:46:32.869963 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:46:32.869963 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:46:32.869963 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:46:32.869963 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:46:32.869963 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:46:32.869963 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:46:32.869963 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:46:32.869963 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:46:32.869963 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:46:32.869963 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:46:32.869963 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 15:46:32.869963 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 15:46:32.869963 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 15:46:32.869963 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 15:46:32.871410 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:46:32.871410 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:46:32.871410 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 15:46:32.871410 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 15:46:32.871410 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 15:46:32.871410 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 15:46:32.871410 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:46:32.871410 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:46:32.871410 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:46:32.871410 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:46:32.871410 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:46:32.871410 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Aug 30 15:46:32.871410 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:46:32.871410 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:46:32.871410 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:46:32.871410 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:46:32.871410 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:46:32.871410 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:46:32.872326 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:46:32.872326 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:46:32.872326 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 15:46:32.872326 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 15:46:32.872326 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 15:46:32.872326 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 15:46:32.872326 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 15:46:32.872326 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 15:46:32.872326 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 30 15:46:32.872326 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Aug 30 15:46:32.872326 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 250/505] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Aug 30 15:46:32 2022] GET /identity/v3/users/glance => generated 84 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 15:46:32.872326 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be8c23d-128f-4f8f-ab74-19ed04d48135 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:32.896476 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4285cb-d0fa-4572-b81b-17a40d95e655 None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:32.929814 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78443f5d-be2e-4c7d-aeb2-c1920a9c2e63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:32.930438 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78443f5d-be2e-4c7d-aeb2-c1920a9c2e63 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:32.930715 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78443f5d-be2e-4c7d-aeb2-c1920a9c2e63 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:32.930983 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78443f5d-be2e-4c7d-aeb2-c1920a9c2e63 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:32.941169 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78443f5d-be2e-4c7d-aeb2-c1920a9c2e63 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 15:46:32.946013 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application [None req-78443f5d-be2e-4c7d-aeb2-c1920a9c2e63 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 15:46:32.946013 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:46:32.946013 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:46:32.946013 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:46:32.946013 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:46:32.946013 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:46:32.946013 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:46:32.946013 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:46:32.946013 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:46:32.946013 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:46:32.946013 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:46:32.946013 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:46:32.946013 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 15:46:32.946013 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 15:46:32.946013 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 15:46:32.946013 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 15:46:32.946918 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 15:46:32.946918 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 15:46:32.946918 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 15:46:32.946918 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 15:46:32.946918 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:46:32.946918 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:46:32.946918 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:46:32.946918 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:46:32.946918 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:46:32.946918 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:46:32.946918 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:46:32.946918 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application with Lock( Aug 30 15:46:32.946918 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:46:32.946918 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:46:32.946918 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:46:32.946918 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:46:32.948289 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:46:32.948289 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:46:32.948289 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:46:32.948289 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:46:32.948289 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 15:46:32.948289 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 15:46:32.948289 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 15:46:32.948289 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 15:46:32.948289 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 15:46:32.948289 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 15:46:32.948289 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 15:46:32.948289 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Aug 30 15:46:32.948289 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 256/506] 10.222.0.20 () {56 vars in 1102 bytes} [Tue Aug 30 15:46:32 2022] GET /identity/v3/roles/service => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 15:46:32.948289 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78443f5d-be2e-4c7d-aeb2-c1920a9c2e63 None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:32.982362 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be8c23d-128f-4f8f-ab74-19ed04d48135 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:33.017422 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e728180-f865-4019-9081-1130c0313144 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:33.018064 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e728180-f865-4019-9081-1130c0313144 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:33.018342 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e728180-f865-4019-9081-1130c0313144 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:33.018616 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e728180-f865-4019-9081-1130c0313144 None admin] PATH_INFO: `/v3/users` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:33.019339 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e728180-f865-4019-9081-1130c0313144 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:46:33.019641 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e728180-f865-4019-9081-1130c0313144 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:33.021074 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e728180-f865-4019-9081-1130c0313144 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:33.057791 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78443f5d-be2e-4c7d-aeb2-c1920a9c2e63 None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:33.058848 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 251/507] 10.222.0.20 () {56 vars in 1133 bytes} [Tue Aug 30 15:46:32 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:46:33.059805 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e728180-f865-4019-9081-1130c0313144 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:33.081385 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:33.093633 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d674b17-f066-45e6-bd40-de1a900c514f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:33.094242 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d674b17-f066-45e6-bd40-de1a900c514f None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:33.094517 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d674b17-f066-45e6-bd40-de1a900c514f None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:33.094793 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d674b17-f066-45e6-bd40-de1a900c514f None admin] PATH_INFO: `/v3/users` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:33.095526 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d674b17-f066-45e6-bd40-de1a900c514f None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81512) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:46:33.095833 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d674b17-f066-45e6-bd40-de1a900c514f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:33.097302 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d674b17-f066-45e6-bd40-de1a900c514f None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:33.105509 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:33.133776 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 257/508] 10.222.0.20 () {56 vars in 1137 bytes} [Tue Aug 30 15:46:32 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:46:33.134650 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d674b17-f066-45e6-bd40-de1a900c514f None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:33.160080 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e728180-f865-4019-9081-1130c0313144 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:33.193552 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb95c5ef-3b94-4a16-98db-60da7af2d6cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:33.194159 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb95c5ef-3b94-4a16-98db-60da7af2d6cc None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:33.194441 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb95c5ef-3b94-4a16-98db-60da7af2d6cc None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:33.194714 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb95c5ef-3b94-4a16-98db-60da7af2d6cc None admin] PATH_INFO: `/v3/users` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:33.195439 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb95c5ef-3b94-4a16-98db-60da7af2d6cc None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:46:33.195743 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb95c5ef-3b94-4a16-98db-60da7af2d6cc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:33.197211 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb95c5ef-3b94-4a16-98db-60da7af2d6cc None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:33.213821 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 252/509] 10.222.0.20 () {56 vars in 1137 bytes} [Tue Aug 30 15:46:33 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:46:33.214644 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb95c5ef-3b94-4a16-98db-60da7af2d6cc None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:33.235834 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d674b17-f066-45e6-bd40-de1a900c514f None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:33.270131 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5813a106-a50b-4c35-8401-c48c0a835a9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:33.270736 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5813a106-a50b-4c35-8401-c48c0a835a9e None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:33.271010 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5813a106-a50b-4c35-8401-c48c0a835a9e None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:33.271278 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5813a106-a50b-4c35-8401-c48c0a835a9e None admin] PATH_INFO: `/v3/roles` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:33.271939 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5813a106-a50b-4c35-8401-c48c0a835a9e None admin] RBAC: Adding query filter params (name=service) {{(pid=81512) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:46:33.272237 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5813a106-a50b-4c35-8401-c48c0a835a9e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:33.273673 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5813a106-a50b-4c35-8401-c48c0a835a9e None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:33.281565 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 258/510] 10.222.0.20 () {56 vars in 1103 bytes} [Tue Aug 30 15:46:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:46:33.282350 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5813a106-a50b-4c35-8401-c48c0a835a9e None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:33.321074 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb95c5ef-3b94-4a16-98db-60da7af2d6cc None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:33.324008 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:33.324784 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:33.355776 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37e1d3d-622c-41a5-9ea9-c7ea1ce30c35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:33.356401 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37e1d3d-622c-41a5-9ea9-c7ea1ce30c35 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:33.356673 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37e1d3d-622c-41a5-9ea9-c7ea1ce30c35 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:33.356936 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37e1d3d-622c-41a5-9ea9-c7ea1ce30c35 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:33.367090 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37e1d3d-622c-41a5-9ea9-c7ea1ce30c35 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:33.368540 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37e1d3d-622c-41a5-9ea9-c7ea1ce30c35 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:33.383190 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-a37e1d3d-622c-41a5-9ea9-c7ea1ce30c35 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 15:46:33.383190 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:46:33.383190 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:46:33.383190 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:46:33.383190 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:46:33.383190 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:46:33.383190 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:46:33.383190 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:46:33.383190 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:46:33.383190 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:46:33.383190 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:46:33.383190 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:46:33.383190 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 15:46:33.383190 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 15:46:33.383190 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 15:46:33.383190 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 15:46:33.384214 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:46:33.384214 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:46:33.384214 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:46:33.384214 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:46:33.384214 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:46:33.384214 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:46:33.384214 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:46:33.384214 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Aug 30 15:46:33.384214 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:46:33.384214 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:46:33.384214 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:46:33.384214 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:46:33.384214 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:46:33.384214 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:46:33.384214 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:46:33.384214 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:46:33.384214 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 15:46:33.385078 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 15:46:33.385078 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 15:46:33.385078 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Aug 30 15:46:33.385467 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5813a106-a50b-4c35-8401-c48c0a835a9e None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:33.385551 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 253/511] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 30 15:46:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 15:46:33.385551 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37e1d3d-622c-41a5-9ea9-c7ea1ce30c35 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:33.418992 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bdaa01-57a9-4b2b-8527-df00133150a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:33.419605 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bdaa01-57a9-4b2b-8527-df00133150a3 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:33.419883 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bdaa01-57a9-4b2b-8527-df00133150a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:33.420155 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bdaa01-57a9-4b2b-8527-df00133150a3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:33.483799 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37e1d3d-622c-41a5-9ea9-c7ea1ce30c35 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:33.490805 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17bdaa01-57a9-4b2b-8527-df00133150a3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:33.492302 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17bdaa01-57a9-4b2b-8527-df00133150a3 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:33.516771 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3f7422-6e62-4266-83dc-841ab4c61fd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:33.517407 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3f7422-6e62-4266-83dc-841ab4c61fd0 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:33.517678 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3f7422-6e62-4266-83dc-841ab4c61fd0 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:33.517920 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3f7422-6e62-4266-83dc-841ab4c61fd0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:33.535375 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:33.545790 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:33.590420 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3f7422-6e62-4266-83dc-841ab4c61fd0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:33.591817 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3f7422-6e62-4266-83dc-841ab4c61fd0 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:33.592742 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 259/512] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Aug 30 15:46:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 15:46:33.594263 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bdaa01-57a9-4b2b-8527-df00133150a3 None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:33.693906 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bdaa01-57a9-4b2b-8527-df00133150a3 None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:33.699820 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 254/513] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Aug 30 15:46:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 15:46:33.700663 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3f7422-6e62-4266-83dc-841ab4c61fd0 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:33.728086 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c4b368-b196-4e8d-8a8b-5481d4744c05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:33.728699 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c4b368-b196-4e8d-8a8b-5481d4744c05 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:33.728978 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c4b368-b196-4e8d-8a8b-5481d4744c05 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:33.729276 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c4b368-b196-4e8d-8a8b-5481d4744c05 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:33.741040 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:33.755973 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:33.797515 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c4b368-b196-4e8d-8a8b-5481d4744c05 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:33.798593 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3f7422-6e62-4266-83dc-841ab4c61fd0 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:33.798967 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c4b368-b196-4e8d-8a8b-5481d4744c05 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:33.833109 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82131347-7624-4704-b06a-9f2b69092231 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:33.833763 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82131347-7624-4704-b06a-9f2b69092231 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:33.834041 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82131347-7624-4704-b06a-9f2b69092231 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:33.834309 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82131347-7624-4704-b06a-9f2b69092231 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:33.844330 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82131347-7624-4704-b06a-9f2b69092231 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:33.845769 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82131347-7624-4704-b06a-9f2b69092231 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:33.860187 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-82131347-7624-4704-b06a-9f2b69092231 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 15:46:33.860187 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:46:33.860187 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:46:33.860187 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:46:33.860187 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:46:33.860187 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:46:33.860187 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:46:33.860187 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:46:33.860187 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:46:33.860187 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:46:33.860187 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:46:33.860187 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:46:33.860187 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 15:46:33.860187 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 15:46:33.860187 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 15:46:33.860187 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 15:46:33.861071 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:46:33.861071 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:46:33.861071 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:46:33.861071 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:46:33.861071 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:46:33.861071 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:46:33.861071 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:46:33.861071 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Aug 30 15:46:33.861071 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:46:33.861071 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:46:33.861071 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:46:33.861071 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:46:33.861071 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:46:33.861071 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:46:33.861071 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:46:33.861071 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:46:33.861071 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 15:46:33.862098 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 15:46:33.862098 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 15:46:33.862098 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Aug 30 15:46:33.862098 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 255/514] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 30 15:46:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 15:46:33.862098 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82131347-7624-4704-b06a-9f2b69092231 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:33.902913 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 260/515] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Aug 30 15:46:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 15:46:33.904020 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c4b368-b196-4e8d-8a8b-5481d4744c05 None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:33.946326 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:33.966211 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82131347-7624-4704-b06a-9f2b69092231 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:33.975075 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:34.001176 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4970c29d-2d54-4061-a5b2-f13072ba543b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:34.001810 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4970c29d-2d54-4061-a5b2-f13072ba543b None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:34.002081 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4970c29d-2d54-4061-a5b2-f13072ba543b None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:34.002344 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4970c29d-2d54-4061-a5b2-f13072ba543b None admin] PATH_INFO: `/v3/domains` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:34.002976 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4970c29d-2d54-4061-a5b2-f13072ba543b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:46:34.003270 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4970c29d-2d54-4061-a5b2-f13072ba543b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:34.004750 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4970c29d-2d54-4061-a5b2-f13072ba543b None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:34.009030 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c4b368-b196-4e8d-8a8b-5481d4744c05 None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:34.016344 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 256/516] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Aug 30 15:46:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:46:34.017160 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4970c29d-2d54-4061-a5b2-f13072ba543b None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:34.045062 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c928d92-cbb6-4b56-94e7-656b45eebb65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:34.045835 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c928d92-cbb6-4b56-94e7-656b45eebb65 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:34.046138 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c928d92-cbb6-4b56-94e7-656b45eebb65 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:34.046408 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c928d92-cbb6-4b56-94e7-656b45eebb65 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:34.056368 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c928d92-cbb6-4b56-94e7-656b45eebb65 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:34.057869 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c928d92-cbb6-4b56-94e7-656b45eebb65 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:34.072193 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application [None req-5c928d92-cbb6-4b56-94e7-656b45eebb65 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 15:46:34.072193 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:46:34.072193 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:46:34.072193 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:46:34.072193 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:46:34.072193 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:46:34.072193 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:46:34.072193 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:46:34.072193 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:46:34.072193 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:46:34.072193 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:46:34.072193 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:46:34.072193 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 15:46:34.072193 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 15:46:34.072193 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 15:46:34.072193 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 15:46:34.073285 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:46:34.073285 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:46:34.073285 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:46:34.073285 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:46:34.073285 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:46:34.073285 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:46:34.073285 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:46:34.073285 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application with Lock( Aug 30 15:46:34.073285 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:46:34.073285 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:46:34.073285 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:46:34.073285 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:46:34.073285 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:46:34.073285 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:46:34.073285 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:46:34.073285 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:46:34.073285 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 15:46:34.074158 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 15:46:34.074158 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 15:46:34.074158 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Aug 30 15:46:34.074158 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 261/517] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 30 15:46:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 15:46:34.074368 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c928d92-cbb6-4b56-94e7-656b45eebb65 None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:34.117268 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4970c29d-2d54-4061-a5b2-f13072ba543b None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:34.152028 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd97c68-68a2-4729-9bc6-759037a4673c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:34.152657 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd97c68-68a2-4729-9bc6-759037a4673c None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:34.152932 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd97c68-68a2-4729-9bc6-759037a4673c None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:34.153207 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd97c68-68a2-4729-9bc6-759037a4673c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:34.164538 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd97c68-68a2-4729-9bc6-759037a4673c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:34.166033 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd97c68-68a2-4729-9bc6-759037a4673c None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:34.166424 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:34.173071 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c928d92-cbb6-4b56-94e7-656b45eebb65 None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:34.181908 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-bdd97c68-68a2-4729-9bc6-759037a4673c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 15:46:34.181908 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:46:34.181908 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:46:34.181908 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:46:34.181908 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:46:34.181908 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:46:34.181908 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:46:34.181908 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:46:34.181908 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:46:34.181908 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:46:34.181908 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:46:34.181908 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:46:34.181908 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 15:46:34.181908 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 15:46:34.181908 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 15:46:34.181908 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 15:46:34.182998 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:46:34.182998 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:46:34.182998 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:46:34.182998 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:46:34.182998 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:46:34.182998 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:46:34.182998 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:46:34.182998 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Aug 30 15:46:34.182998 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:46:34.182998 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:46:34.182998 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:46:34.182998 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:46:34.182998 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:46:34.182998 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:46:34.182998 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:46:34.182998 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:46:34.182998 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 15:46:34.183881 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 15:46:34.183881 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 15:46:34.183881 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Aug 30 15:46:34.183881 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 257/518] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 30 15:46:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 15:46:34.183881 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd97c68-68a2-4729-9bc6-759037a4673c None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:34.208888 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:34.210999 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c37882-34b4-4815-a3f5-fc20fd9ed2f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:34.211925 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c37882-34b4-4815-a3f5-fc20fd9ed2f5 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:34.212292 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c37882-34b4-4815-a3f5-fc20fd9ed2f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:34.212626 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c37882-34b4-4815-a3f5-fc20fd9ed2f5 None admin] PATH_INFO: `/v3/domains` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:34.213379 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c37882-34b4-4815-a3f5-fc20fd9ed2f5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81512) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:46:34.213756 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c37882-34b4-4815-a3f5-fc20fd9ed2f5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:34.215244 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c37882-34b4-4815-a3f5-fc20fd9ed2f5 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:34.227586 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 262/519] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Aug 30 15:46:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:46:34.228363 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c37882-34b4-4815-a3f5-fc20fd9ed2f5 None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:34.291086 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd97c68-68a2-4729-9bc6-759037a4673c None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:34.324196 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749430c2-887f-43c7-9bc2-d6baafd8e5ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:34.324825 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749430c2-887f-43c7-9bc2-d6baafd8e5ca None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:34.325097 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749430c2-887f-43c7-9bc2-d6baafd8e5ca None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:34.325389 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749430c2-887f-43c7-9bc2-d6baafd8e5ca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:34.327302 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c37882-34b4-4815-a3f5-fc20fd9ed2f5 None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:34.335751 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749430c2-887f-43c7-9bc2-d6baafd8e5ca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:34.337212 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749430c2-887f-43c7-9bc2-d6baafd8e5ca None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:34.351047 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-749430c2-887f-43c7-9bc2-d6baafd8e5ca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 15:46:34.351047 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:46:34.351047 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:46:34.351047 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:46:34.351047 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:46:34.351047 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:46:34.351047 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:46:34.351047 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:46:34.351047 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:46:34.351047 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:46:34.351047 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:46:34.351047 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:46:34.351047 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 15:46:34.351047 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 15:46:34.351047 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 15:46:34.351047 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 15:46:34.352097 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:46:34.352097 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:46:34.352097 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:46:34.352097 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:46:34.352097 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:46:34.352097 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:46:34.352097 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:46:34.352097 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Aug 30 15:46:34.352097 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:46:34.352097 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:46:34.352097 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:46:34.352097 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:46:34.352097 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:46:34.352097 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:46:34.352097 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:46:34.352097 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:46:34.352097 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 15:46:34.353060 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 15:46:34.353060 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 15:46:34.353060 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Aug 30 15:46:34.353060 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 258/520] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 30 15:46:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 15:46:34.353453 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749430c2-887f-43c7-9bc2-d6baafd8e5ca None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:34.365748 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a2713d-1376-47c8-936c-31cba4782dbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:34.366137 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a2713d-1376-47c8-936c-31cba4782dbe None admin] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:34.366392 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a2713d-1376-47c8-936c-31cba4782dbe None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:34.366660 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a2713d-1376-47c8-936c-31cba4782dbe None admin] PATH_INFO: `/v3/users` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:34.367367 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a2713d-1376-47c8-936c-31cba4782dbe None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:34.368777 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a2713d-1376-47c8-936c-31cba4782dbe None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:34.393165 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 263/521] 10.222.0.20 () {60 vars in 1132 bytes} [Tue Aug 30 15:46:34 2022] POST /identity/v3/users => generated 247 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 15:46:34.394480 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a2713d-1376-47c8-936c-31cba4782dbe None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:34.409803 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:34.429558 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:34.463842 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749430c2-887f-43c7-9bc2-d6baafd8e5ca None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:34.502345 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0acbbf-1de6-45b0-906f-d5d38075a809 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:34.502980 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0acbbf-1de6-45b0-906f-d5d38075a809 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:34.503260 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0acbbf-1de6-45b0-906f-d5d38075a809 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:34.503526 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0acbbf-1de6-45b0-906f-d5d38075a809 None admin] PATH_INFO: `/v3/domains` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:34.504193 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0acbbf-1de6-45b0-906f-d5d38075a809 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:46:34.504499 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0acbbf-1de6-45b0-906f-d5d38075a809 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:34.505947 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0acbbf-1de6-45b0-906f-d5d38075a809 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:34.513023 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a2713d-1376-47c8-936c-31cba4782dbe None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:34.521562 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 259/522] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Aug 30 15:46:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:46:34.522408 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0acbbf-1de6-45b0-906f-d5d38075a809 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:34.555816 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a997edb9-6e79-4133-aa09-88ab1f27321c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:34.556833 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a997edb9-6e79-4133-aa09-88ab1f27321c None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:34.556833 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a997edb9-6e79-4133-aa09-88ab1f27321c None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:34.557026 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a997edb9-6e79-4133-aa09-88ab1f27321c None admin] PATH_INFO: `/v3/domains` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:34.557729 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a997edb9-6e79-4133-aa09-88ab1f27321c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81512) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:46:34.559015 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a997edb9-6e79-4133-aa09-88ab1f27321c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:34.559456 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a997edb9-6e79-4133-aa09-88ab1f27321c None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:34.571956 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 264/523] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Aug 30 15:46:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:46:34.572800 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a997edb9-6e79-4133-aa09-88ab1f27321c None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:34.649105 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0acbbf-1de6-45b0-906f-d5d38075a809 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:34.649324 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:34.681037 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:34.686531 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a997edb9-6e79-4133-aa09-88ab1f27321c None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:34.686657 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7471344-9268-494f-bea1-2912e8d0e3e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:34.687381 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7471344-9268-494f-bea1-2912e8d0e3e6 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:34.687724 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7471344-9268-494f-bea1-2912e8d0e3e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:34.688061 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7471344-9268-494f-bea1-2912e8d0e3e6 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:34.695382 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7471344-9268-494f-bea1-2912e8d0e3e6 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:34.696930 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7471344-9268-494f-bea1-2912e8d0e3e6 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:34.709904 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-b7471344-9268-494f-bea1-2912e8d0e3e6 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 30 15:46:34.709904 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:46:34.709904 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:46:34.709904 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:46:34.709904 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:46:34.709904 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:46:34.709904 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:46:34.709904 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:46:34.709904 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:46:34.709904 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:46:34.709904 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:46:34.709904 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:46:34.709904 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 15:46:34.709904 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 15:46:34.709904 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 15:46:34.709904 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 15:46:34.711463 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:46:34.711463 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:46:34.711463 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 15:46:34.711463 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 15:46:34.711463 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 15:46:34.711463 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 15:46:34.711463 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:46:34.711463 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:46:34.711463 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:46:34.711463 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:46:34.711463 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:46:34.711463 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Aug 30 15:46:34.711463 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:46:34.711463 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:46:34.711463 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:46:34.711463 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:46:34.711463 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:46:34.711463 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:46:34.712362 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:46:34.712362 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:46:34.712362 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 15:46:34.712362 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 15:46:34.712362 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 15:46:34.712362 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 15:46:34.712362 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 15:46:34.712362 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 15:46:34.712362 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 30 15:46:34.712362 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Aug 30 15:46:34.712362 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 260/524] 10.222.0.20 () {56 vars in 1102 bytes} [Tue Aug 30 15:46:34 2022] GET /identity/v3/users/neutron => generated 85 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 15:46:34.712362 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7471344-9268-494f-bea1-2912e8d0e3e6 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:34.731676 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44e98dd-7713-4b68-9b0c-aa855b04ef00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:34.732290 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44e98dd-7713-4b68-9b0c-aa855b04ef00 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:34.732622 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44e98dd-7713-4b68-9b0c-aa855b04ef00 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:34.732900 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44e98dd-7713-4b68-9b0c-aa855b04ef00 None admin] PATH_INFO: `/v3/domains` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:34.733651 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44e98dd-7713-4b68-9b0c-aa855b04ef00 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81512) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:46:34.733956 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44e98dd-7713-4b68-9b0c-aa855b04ef00 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:34.735384 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44e98dd-7713-4b68-9b0c-aa855b04ef00 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:34.747913 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 265/525] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Aug 30 15:46:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:46:34.747913 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44e98dd-7713-4b68-9b0c-aa855b04ef00 None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:34.822488 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7471344-9268-494f-bea1-2912e8d0e3e6 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:34.854294 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44e98dd-7713-4b68-9b0c-aa855b04ef00 None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:34.856488 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44661514-c612-4699-b4d7-dbd094cea0a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:34.857133 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44661514-c612-4699-b4d7-dbd094cea0a2 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:34.857508 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44661514-c612-4699-b4d7-dbd094cea0a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:34.857782 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44661514-c612-4699-b4d7-dbd094cea0a2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:34.864864 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44661514-c612-4699-b4d7-dbd094cea0a2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:34.866353 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44661514-c612-4699-b4d7-dbd094cea0a2 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:34.878648 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-44661514-c612-4699-b4d7-dbd094cea0a2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 15:46:34.878648 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:46:34.878648 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:46:34.878648 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:46:34.878648 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:46:34.878648 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:46:34.878648 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:46:34.878648 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:46:34.878648 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:46:34.878648 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:46:34.878648 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:46:34.878648 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:46:34.878648 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 15:46:34.878648 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 15:46:34.878648 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 15:46:34.879619 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 15:46:34.879619 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:46:34.879619 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:46:34.879619 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:46:34.879619 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:46:34.879619 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:46:34.879619 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:46:34.879619 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:46:34.879619 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Aug 30 15:46:34.879619 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:46:34.879619 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:46:34.879619 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:46:34.879619 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:46:34.879619 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:46:34.879619 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:46:34.879619 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:46:34.879619 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:46:34.880485 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 15:46:34.880485 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 15:46:34.880485 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 15:46:34.880485 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 15:46:34.880485 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 15:46:34.880485 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 15:46:34.880485 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 15:46:34.880485 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Aug 30 15:46:34.880485 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 261/526] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Aug 30 15:46:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 15:46:34.880964 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44661514-c612-4699-b4d7-dbd094cea0a2 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:34.890375 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:34.897030 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4d3864-ec1c-48d1-97f8-b6f070e0d303 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:34.897632 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4d3864-ec1c-48d1-97f8-b6f070e0d303 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:34.897914 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4d3864-ec1c-48d1-97f8-b6f070e0d303 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:34.898186 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4d3864-ec1c-48d1-97f8-b6f070e0d303 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:34.905493 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4d3864-ec1c-48d1-97f8-b6f070e0d303 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:34.907005 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4d3864-ec1c-48d1-97f8-b6f070e0d303 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:34.919201 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application [None req-da4d3864-ec1c-48d1-97f8-b6f070e0d303 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 15:46:34.919201 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:46:34.919201 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:46:34.919201 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:46:34.919201 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:46:34.919201 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:46:34.919201 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:46:34.919201 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:46:34.919201 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:46:34.919201 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:46:34.919201 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:46:34.919201 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:46:34.919201 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 15:46:34.919201 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 15:46:34.919201 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 15:46:34.920251 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 15:46:34.920251 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:46:34.920251 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:46:34.920251 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:46:34.920251 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:46:34.920251 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:46:34.920251 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:46:34.920251 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:46:34.920251 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application with Lock( Aug 30 15:46:34.920251 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:46:34.920251 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:46:34.920251 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:46:34.920251 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:46:34.920251 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:46:34.920251 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:46:34.920251 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:46:34.920251 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:46:34.921535 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 15:46:34.921535 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 15:46:34.921535 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 15:46:34.921535 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 15:46:34.921535 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 15:46:34.921535 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 15:46:34.921535 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 15:46:34.921535 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Aug 30 15:46:34.925208 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 266/527] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Aug 30 15:46:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 15:46:34.926074 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4d3864-ec1c-48d1-97f8-b6f070e0d303 None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:34.939995 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:34.991121 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44661514-c612-4699-b4d7-dbd094cea0a2 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:35.026598 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e84808-9500-4c3f-b83d-66c713077f36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:35.027215 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e84808-9500-4c3f-b83d-66c713077f36 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:35.027494 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e84808-9500-4c3f-b83d-66c713077f36 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:35.027763 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e84808-9500-4c3f-b83d-66c713077f36 None admin] PATH_INFO: `/v3/users` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:35.028503 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e84808-9500-4c3f-b83d-66c713077f36 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:46:35.028815 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e84808-9500-4c3f-b83d-66c713077f36 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:35.030331 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e84808-9500-4c3f-b83d-66c713077f36 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:35.034854 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4d3864-ec1c-48d1-97f8-b6f070e0d303 None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:35.047952 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 262/528] 10.222.0.20 () {56 vars in 1139 bytes} [Tue Aug 30 15:46:34 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:46:35.048783 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e84808-9500-4c3f-b83d-66c713077f36 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:35.081450 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06420ae2-8981-477a-908f-3c64a1676b5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:35.083177 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06420ae2-8981-477a-908f-3c64a1676b5f None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:35.083660 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06420ae2-8981-477a-908f-3c64a1676b5f None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:35.084159 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06420ae2-8981-477a-908f-3c64a1676b5f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:35.092284 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06420ae2-8981-477a-908f-3c64a1676b5f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:35.094851 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06420ae2-8981-477a-908f-3c64a1676b5f None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:35.109895 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application [None req-06420ae2-8981-477a-908f-3c64a1676b5f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 15:46:35.109895 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:46:35.109895 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:46:35.109895 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:46:35.109895 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:46:35.109895 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:46:35.109895 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:46:35.109895 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:46:35.109895 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:46:35.109895 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:46:35.109895 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:46:35.109895 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:46:35.109895 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 15:46:35.109895 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 15:46:35.109895 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 15:46:35.110731 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 15:46:35.110731 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:46:35.110731 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:46:35.110731 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:46:35.110731 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:46:35.110731 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:46:35.110731 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:46:35.110731 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:46:35.110731 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application with Lock( Aug 30 15:46:35.110731 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:46:35.110731 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:46:35.110731 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:46:35.110731 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:46:35.110731 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:46:35.110731 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:46:35.110731 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:46:35.110731 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:46:35.111593 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 15:46:35.111593 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 15:46:35.111593 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 15:46:35.111593 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 15:46:35.111593 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 15:46:35.111593 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 15:46:35.111593 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 15:46:35.111593 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Aug 30 15:46:35.117867 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 267/529] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Aug 30 15:46:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 15:46:35.117867 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06420ae2-8981-477a-908f-3c64a1676b5f None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:35.151934 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e84808-9500-4c3f-b83d-66c713077f36 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:35.163327 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:35.165333 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:35.191581 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08461c8-611f-436a-9230-ea2ccb3feb64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:35.192015 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08461c8-611f-436a-9230-ea2ccb3feb64 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:35.192294 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08461c8-611f-436a-9230-ea2ccb3feb64 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:35.192566 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08461c8-611f-436a-9230-ea2ccb3feb64 None admin] PATH_INFO: `/v3/projects` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:35.193287 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08461c8-611f-436a-9230-ea2ccb3feb64 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:46:35.193850 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08461c8-611f-436a-9230-ea2ccb3feb64 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:35.195032 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08461c8-611f-436a-9230-ea2ccb3feb64 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:35.217955 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 263/530] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Aug 30 15:46:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:46:35.217955 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08461c8-611f-436a-9230-ea2ccb3feb64 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:35.222421 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06420ae2-8981-477a-908f-3c64a1676b5f None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:35.257717 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18b5dfd-4d8e-4bcb-b4a6-b3bee1940df8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:35.258326 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18b5dfd-4d8e-4bcb-b4a6-b3bee1940df8 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:35.258601 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18b5dfd-4d8e-4bcb-b4a6-b3bee1940df8 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:35.258866 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18b5dfd-4d8e-4bcb-b4a6-b3bee1940df8 None admin] PATH_INFO: `/v3/projects` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:35.259557 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18b5dfd-4d8e-4bcb-b4a6-b3bee1940df8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81512) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:46:35.259858 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18b5dfd-4d8e-4bcb-b4a6-b3bee1940df8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:35.261339 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18b5dfd-4d8e-4bcb-b4a6-b3bee1940df8 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:35.279135 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 268/531] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Aug 30 15:46:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:46:35.289674 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18b5dfd-4d8e-4bcb-b4a6-b3bee1940df8 None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:35.322823 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08461c8-611f-436a-9230-ea2ccb3feb64 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:35.360302 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc9b9a5-25b0-42e9-ae5f-e5e3cebd02bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:35.361006 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc9b9a5-25b0-42e9-ae5f-e5e3cebd02bb None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:35.361313 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc9b9a5-25b0-42e9-ae5f-e5e3cebd02bb None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:35.361586 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc9b9a5-25b0-42e9-ae5f-e5e3cebd02bb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:35.393073 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18b5dfd-4d8e-4bcb-b4a6-b3bee1940df8 None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:35.400856 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:35.405818 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:35.429795 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8793ea-416c-460e-a3b6-acfdc085c40d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:35.430405 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8793ea-416c-460e-a3b6-acfdc085c40d None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:35.430681 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8793ea-416c-460e-a3b6-acfdc085c40d None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:35.430949 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8793ea-416c-460e-a3b6-acfdc085c40d None admin] PATH_INFO: `/v3/projects` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:35.431641 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8793ea-416c-460e-a3b6-acfdc085c40d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81512) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:46:35.431948 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8793ea-416c-460e-a3b6-acfdc085c40d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:35.433427 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8793ea-416c-460e-a3b6-acfdc085c40d None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:35.433532 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc9b9a5-25b0-42e9-ae5f-e5e3cebd02bb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:35.435034 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc9b9a5-25b0-42e9-ae5f-e5e3cebd02bb None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:35.445647 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 269/532] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Aug 30 15:46:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:46:35.446350 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8793ea-416c-460e-a3b6-acfdc085c40d None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:35.551377 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8793ea-416c-460e-a3b6-acfdc085c40d None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:35.556367 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 264/533] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Aug 30 15:46:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 15:46:35.556367 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc9b9a5-25b0-42e9-ae5f-e5e3cebd02bb None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:35.591048 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27360c4e-8f72-42c7-acfc-b68cc2798a55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:35.591746 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27360c4e-8f72-42c7-acfc-b68cc2798a55 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:35.592088 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27360c4e-8f72-42c7-acfc-b68cc2798a55 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:35.592420 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27360c4e-8f72-42c7-acfc-b68cc2798a55 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:35.593254 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27360c4e-8f72-42c7-acfc-b68cc2798a55 None admin] RBAC: Adding query filter params (role.id=57eb8466096346b39319536c6feeb6f7, scope.project.id=854fe71b49224dff9a4b1766a20ab266, user.id=7a1e756ae9be4a7bac1e81acf07e8f3b) {{(pid=81512) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:46:35.593631 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27360c4e-8f72-42c7-acfc-b68cc2798a55 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:35.595174 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27360c4e-8f72-42c7-acfc-b68cc2798a55 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:35.600855 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 270/534] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Aug 30 15:46:35 2022] GET /identity/v3/role_assignments?user.id=7a1e756ae9be4a7bac1e81acf07e8f3b&scope.project.id=854fe71b49224dff9a4b1766a20ab266&role.id=57eb8466096346b39319536c6feeb6f7 => generated 259 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:46:35.601754 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27360c4e-8f72-42c7-acfc-b68cc2798a55 None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:35.645584 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:35.654583 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:35.689771 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc9b9a5-25b0-42e9-ae5f-e5e3cebd02bb None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:35.731345 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27360c4e-8f72-42c7-acfc-b68cc2798a55 None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:35.733234 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992446de-7771-49ce-af25-e693342720b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:35.734045 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992446de-7771-49ce-af25-e693342720b3 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:35.734340 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992446de-7771-49ce-af25-e693342720b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:35.734615 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992446de-7771-49ce-af25-e693342720b3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:35.735366 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992446de-7771-49ce-af25-e693342720b3 None admin] RBAC: Adding query filter params (role.id=57eb8466096346b39319536c6feeb6f7, scope.project.id=854fe71b49224dff9a4b1766a20ab266, user.id=0100a04821804993866ea20f159663f9) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:46:35.735683 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992446de-7771-49ce-af25-e693342720b3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:35.737187 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992446de-7771-49ce-af25-e693342720b3 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:35.744035 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 265/535] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Aug 30 15:46:35 2022] GET /identity/v3/role_assignments?user.id=0100a04821804993866ea20f159663f9&scope.project.id=854fe71b49224dff9a4b1766a20ab266&role.id=57eb8466096346b39319536c6feeb6f7 => generated 259 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:46:35.745372 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992446de-7771-49ce-af25-e693342720b3 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:35.793764 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3e27c9-5b18-4f40-9180-45255e7f65e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:35.795238 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3e27c9-5b18-4f40-9180-45255e7f65e4 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:35.795738 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3e27c9-5b18-4f40-9180-45255e7f65e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:35.796336 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3e27c9-5b18-4f40-9180-45255e7f65e4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:35.799244 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3e27c9-5b18-4f40-9180-45255e7f65e4 None admin] RBAC: Adding query filter params (role.id=57eb8466096346b39319536c6feeb6f7, scope.project.id=854fe71b49224dff9a4b1766a20ab266, user.id=b70e06a7d06d4199ab4fc36342ee1d68) {{(pid=81512) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:46:35.799923 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3e27c9-5b18-4f40-9180-45255e7f65e4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:35.801755 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3e27c9-5b18-4f40-9180-45255e7f65e4 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:35.814513 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 271/536] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Aug 30 15:46:35 2022] GET /identity/v3/role_assignments?user.id=b70e06a7d06d4199ab4fc36342ee1d68&scope.project.id=854fe71b49224dff9a4b1766a20ab266&role.id=57eb8466096346b39319536c6feeb6f7 => generated 259 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:46:35.889098 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992446de-7771-49ce-af25-e693342720b3 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:35.928867 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7929ff2b-ac0e-4a0e-af92-26450b54a8f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:35.929569 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7929ff2b-ac0e-4a0e-af92-26450b54a8f8 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:35.929856 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7929ff2b-ac0e-4a0e-af92-26450b54a8f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:35.930130 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7929ff2b-ac0e-4a0e-af92-26450b54a8f8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:35.932360 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:35.942355 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7929ff2b-ac0e-4a0e-af92-26450b54a8f8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:35.943793 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7929ff2b-ac0e-4a0e-af92-26450b54a8f8 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:35.959822 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-7929ff2b-ac0e-4a0e-af92-26450b54a8f8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 15:46:35.959822 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:46:35.959822 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:46:35.959822 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:46:35.959822 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:46:35.959822 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:46:35.959822 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:46:35.959822 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:46:35.959822 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:46:35.959822 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:46:35.959822 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:46:35.959822 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:46:35.959822 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 15:46:35.959822 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 15:46:35.959822 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 15:46:35.959822 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 15:46:35.961285 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:46:35.961285 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:46:35.961285 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:46:35.961285 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:46:35.961285 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:46:35.961285 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:46:35.961285 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:46:35.961285 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Aug 30 15:46:35.961285 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:46:35.961285 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:46:35.961285 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:46:35.961285 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:46:35.961285 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:46:35.961285 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:46:35.961285 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:46:35.961285 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:46:35.961285 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 15:46:35.962510 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 15:46:35.962510 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 15:46:35.962510 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Aug 30 15:46:35.964299 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 266/537] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 30 15:46:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 15:46:35.969826 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3e27c9-5b18-4f40-9180-45255e7f65e4 None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:36.076991 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:36.081326 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3e27c9-5b18-4f40-9180-45255e7f65e4 None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:36.117644 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60929365-6397-43d3-bcf6-0834e514d08d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:36.118497 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60929365-6397-43d3-bcf6-0834e514d08d None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:36.118939 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60929365-6397-43d3-bcf6-0834e514d08d None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:36.119360 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60929365-6397-43d3-bcf6-0834e514d08d None admin] PATH_INFO: `/v3/domains` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:36.120147 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60929365-6397-43d3-bcf6-0834e514d08d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81512) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:46:36.120604 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60929365-6397-43d3-bcf6-0834e514d08d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:36.122500 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60929365-6397-43d3-bcf6-0834e514d08d None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:36.135545 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 272/538] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Aug 30 15:46:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 15:46:36.142339 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7929ff2b-ac0e-4a0e-af92-26450b54a8f8 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:36.281092 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7929ff2b-ac0e-4a0e-af92-26450b54a8f8 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:36.320640 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32372169-7c00-45e9-991d-214e0bd1932b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:36.321311 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32372169-7c00-45e9-991d-214e0bd1932b None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:36.321592 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32372169-7c00-45e9-991d-214e0bd1932b None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:36.321862 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32372169-7c00-45e9-991d-214e0bd1932b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:36.328801 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32372169-7c00-45e9-991d-214e0bd1932b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:36.330303 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32372169-7c00-45e9-991d-214e0bd1932b None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:36.341948 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-32372169-7c00-45e9-991d-214e0bd1932b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 15:46:36.341948 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:46:36.341948 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:46:36.341948 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:46:36.341948 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:46:36.341948 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:46:36.341948 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:46:36.341948 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:46:36.341948 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:46:36.341948 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:46:36.341948 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:46:36.341948 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:46:36.341948 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 15:46:36.341948 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 15:46:36.341948 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 15:46:36.342909 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 15:46:36.342909 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:46:36.342909 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:46:36.342909 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:46:36.342909 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:46:36.342909 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:46:36.342909 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:46:36.342909 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:46:36.342909 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Aug 30 15:46:36.342909 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:46:36.342909 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:46:36.342909 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:46:36.342909 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:46:36.342909 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:46:36.342909 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:46:36.342909 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:46:36.342909 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:46:36.343796 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 15:46:36.343796 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 15:46:36.343796 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 15:46:36.343796 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 15:46:36.343796 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 15:46:36.343796 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 15:46:36.343796 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 15:46:36.343796 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Aug 30 15:46:36.343796 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 267/539] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Aug 30 15:46:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 15:46:36.350597 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60929365-6397-43d3-bcf6-0834e514d08d None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:36.471045 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60929365-6397-43d3-bcf6-0834e514d08d None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:36.508057 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f9eaab-773b-468c-b117-5f3a94d4d393 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:36.509110 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f9eaab-773b-468c-b117-5f3a94d4d393 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:36.509610 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f9eaab-773b-468c-b117-5f3a94d4d393 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:36.510072 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f9eaab-773b-468c-b117-5f3a94d4d393 None admin] PATH_INFO: `/v3/projects` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:36.510956 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f9eaab-773b-468c-b117-5f3a94d4d393 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81512) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:46:36.511454 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f9eaab-773b-468c-b117-5f3a94d4d393 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:36.513098 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f9eaab-773b-468c-b117-5f3a94d4d393 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:36.526150 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 273/540] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Aug 30 15:46:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:46:36.533314 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32372169-7c00-45e9-991d-214e0bd1932b None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:36.577927 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1930989-ad81-4495-9ca3-a1a115d3c5df None None] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:36.578232 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1930989-ad81-4495-9ca3-a1a115d3c5df None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:36.578465 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1930989-ad81-4495-9ca3-a1a115d3c5df None None] PATH_INFO: `/` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:36.580042 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 274/541] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 30 15:46:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 15:46:36.584922 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a137b6-8e9a-47c2-b752-51a13845985d None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:36.585199 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a137b6-8e9a-47c2-b752-51a13845985d None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:36.585547 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a137b6-8e9a-47c2-b752-51a13845985d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:36.600212 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:36.622223 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-a8a137b6-8e9a-47c2-b752-51a13845985d None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:46:36.623449 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-a8a137b6-8e9a-47c2-b752-51a13845985d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:46:36.655138 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:36.655413 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32372169-7c00-45e9-991d-214e0bd1932b None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:36.689625 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57d800c-c1f6-482f-ba6a-1b150942afa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:36.690233 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57d800c-c1f6-482f-ba6a-1b150942afa0 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:36.691192 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57d800c-c1f6-482f-ba6a-1b150942afa0 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:36.691192 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57d800c-c1f6-482f-ba6a-1b150942afa0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:36.691483 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57d800c-c1f6-482f-ba6a-1b150942afa0 None admin] RBAC: Adding query filter params (role.id=57eb8466096346b39319536c6feeb6f7, scope.project.id=854fe71b49224dff9a4b1766a20ab266, user.id=65f4157105f3488f807b7475008ab722) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:46:36.691788 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57d800c-c1f6-482f-ba6a-1b150942afa0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:36.693260 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57d800c-c1f6-482f-ba6a-1b150942afa0 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:36.702197 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 268/542] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Aug 30 15:46:36 2022] GET /identity/v3/role_assignments?user.id=65f4157105f3488f807b7475008ab722&scope.project.id=854fe71b49224dff9a4b1766a20ab266&role.id=57eb8466096346b39319536c6feeb6f7 => generated 259 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:46:36.736060 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 275/543] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 30 15:46:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 15:46:37.057443 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57d800c-c1f6-482f-ba6a-1b150942afa0 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:37.179852 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57d800c-c1f6-482f-ba6a-1b150942afa0 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:37.193069 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:37.229936 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0828fe0b-3e80-4278-b560-2ffca113c922 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:37.230958 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0828fe0b-3e80-4278-b560-2ffca113c922 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:37.231432 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0828fe0b-3e80-4278-b560-2ffca113c922 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:37.231929 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0828fe0b-3e80-4278-b560-2ffca113c922 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:37.246351 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0828fe0b-3e80-4278-b560-2ffca113c922 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 15:46:37.252112 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-0828fe0b-3e80-4278-b560-2ffca113c922 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 15:46:37.252112 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:46:37.252112 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:46:37.252112 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:46:37.252112 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:46:37.252112 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:46:37.252112 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:46:37.252112 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:46:37.252112 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:46:37.252112 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:46:37.252112 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:46:37.252112 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:46:37.252112 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 15:46:37.252112 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 15:46:37.252112 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 15:46:37.252112 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 15:46:37.253001 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 15:46:37.253001 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 15:46:37.253001 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 15:46:37.253001 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 15:46:37.253001 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:46:37.253001 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:46:37.253001 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:46:37.253001 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:46:37.253001 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:46:37.253001 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:46:37.253001 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:46:37.253001 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Aug 30 15:46:37.253001 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:46:37.253001 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:46:37.253001 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:46:37.253001 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:46:37.253842 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:46:37.253842 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:46:37.253842 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:46:37.253842 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:46:37.253842 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 15:46:37.253842 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 15:46:37.253842 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 15:46:37.253842 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 15:46:37.253842 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 15:46:37.253842 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 15:46:37.253842 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 15:46:37.253842 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Aug 30 15:46:37.256263 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 269/544] 10.222.0.20 () {56 vars in 1102 bytes} [Tue Aug 30 15:46:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 15:46:37.262342 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a137b6-8e9a-47c2-b752-51a13845985d None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:37.376224 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a137b6-8e9a-47c2-b752-51a13845985d None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:37.416664 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27ff81a-37e4-4f4f-8557-c42c90d021a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:37.417693 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27ff81a-37e4-4f4f-8557-c42c90d021a1 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:37.418166 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27ff81a-37e4-4f4f-8557-c42c90d021a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:37.418646 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27ff81a-37e4-4f4f-8557-c42c90d021a1 None admin] PATH_INFO: `/v3/roles` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:37.419500 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27ff81a-37e4-4f4f-8557-c42c90d021a1 None admin] RBAC: Adding query filter params (name=service) {{(pid=81512) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:46:37.419991 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27ff81a-37e4-4f4f-8557-c42c90d021a1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:37.421764 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27ff81a-37e4-4f4f-8557-c42c90d021a1 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:37.430949 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 276/545] 10.222.0.20 () {56 vars in 1103 bytes} [Tue Aug 30 15:46:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:46:37.436102 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0828fe0b-3e80-4278-b560-2ffca113c922 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:37.571045 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0828fe0b-3e80-4278-b560-2ffca113c922 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:37.621936 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a138d3-fec8-406b-acc0-e54252c10a16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:37.622579 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a138d3-fec8-406b-acc0-e54252c10a16 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:37.622857 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a138d3-fec8-406b-acc0-e54252c10a16 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:37.623131 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a138d3-fec8-406b-acc0-e54252c10a16 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:37.636904 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a138d3-fec8-406b-acc0-e54252c10a16 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:37.638487 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a138d3-fec8-406b-acc0-e54252c10a16 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:37.661333 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-32a138d3-fec8-406b-acc0-e54252c10a16 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 15:46:37.661333 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:46:37.661333 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:46:37.661333 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:46:37.661333 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:46:37.661333 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:46:37.661333 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:46:37.661333 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:46:37.661333 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:46:37.661333 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:46:37.661333 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:46:37.661333 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:46:37.661333 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 15:46:37.661333 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 15:46:37.661333 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 15:46:37.661333 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 15:46:37.662298 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:46:37.662298 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:46:37.662298 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:46:37.662298 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:46:37.662298 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:46:37.662298 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:46:37.662298 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:46:37.662298 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Aug 30 15:46:37.662298 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:46:37.662298 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:46:37.662298 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:46:37.662298 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:46:37.662298 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:46:37.662298 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:46:37.662298 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:46:37.662298 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:46:37.662298 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 15:46:37.663555 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 15:46:37.663555 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 15:46:37.663555 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Aug 30 15:46:37.663555 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 270/546] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 30 15:46:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 15:46:37.668470 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27ff81a-37e4-4f4f-8557-c42c90d021a1 None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:37.693139 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:37.793538 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27ff81a-37e4-4f4f-8557-c42c90d021a1 None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:37.832685 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c8edc3-7917-4123-8622-bd37fde27c03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:37.833324 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c8edc3-7917-4123-8622-bd37fde27c03 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:37.833603 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c8edc3-7917-4123-8622-bd37fde27c03 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:37.833876 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c8edc3-7917-4123-8622-bd37fde27c03 None admin] PATH_INFO: `/v3/domains` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:37.834515 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c8edc3-7917-4123-8622-bd37fde27c03 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81512) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:46:37.834813 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c8edc3-7917-4123-8622-bd37fde27c03 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:37.836201 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c8edc3-7917-4123-8622-bd37fde27c03 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:37.848476 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 277/547] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Aug 30 15:46:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:46:37.858379 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a138d3-fec8-406b-acc0-e54252c10a16 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:37.900078 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:37.975322 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95110b3-c006-49c9-bda1-355637a1e38e None None] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:37.975655 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95110b3-c006-49c9-bda1-355637a1e38e None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:37.975874 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95110b3-c006-49c9-bda1-355637a1e38e None None] PATH_INFO: `/` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:37.977564 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 278/548] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 30 15:46:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 15:46:37.982799 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740df898-6f97-4df8-9a81-7dfa94f81341 None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:37.983081 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740df898-6f97-4df8-9a81-7dfa94f81341 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:37.983413 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740df898-6f97-4df8-9a81-7dfa94f81341 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:38.003146 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-740df898-6f97-4df8-9a81-7dfa94f81341 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:46:38.004396 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-740df898-6f97-4df8-9a81-7dfa94f81341 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:46:38.008942 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a138d3-fec8-406b-acc0-e54252c10a16 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:38.074737 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c66f53-cb77-433f-85f8-4a1cbbca30a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:38.075520 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c66f53-cb77-433f-85f8-4a1cbbca30a2 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:38.075712 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c66f53-cb77-433f-85f8-4a1cbbca30a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:38.075991 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c66f53-cb77-433f-85f8-4a1cbbca30a2 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:38.088891 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c66f53-cb77-433f-85f8-4a1cbbca30a2 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:38.091139 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c66f53-cb77-433f-85f8-4a1cbbca30a2 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:38.096597 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:38.107134 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-a2c66f53-cb77-433f-85f8-4a1cbbca30a2 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 30 15:46:38.107134 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:46:38.107134 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:46:38.107134 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:46:38.107134 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:46:38.107134 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:46:38.107134 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:46:38.107134 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:46:38.107134 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:46:38.107134 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:46:38.107134 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:46:38.107134 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:46:38.107134 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 15:46:38.107134 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 15:46:38.107134 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 15:46:38.107134 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 15:46:38.109277 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:46:38.109277 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:46:38.109277 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 15:46:38.109277 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 15:46:38.109277 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 15:46:38.109277 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 15:46:38.109277 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:46:38.109277 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:46:38.109277 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:46:38.109277 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:46:38.109277 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:46:38.109277 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Aug 30 15:46:38.109277 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:46:38.109277 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:46:38.109277 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:46:38.109277 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:46:38.109277 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:46:38.109277 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:46:38.110827 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:46:38.110827 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:46:38.110827 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 15:46:38.110827 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 15:46:38.110827 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 15:46:38.110827 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 15:46:38.110827 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 15:46:38.110827 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 15:46:38.110827 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 30 15:46:38.110827 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Aug 30 15:46:38.111353 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 279/549] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 30 15:46:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 15:46:38.113991 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 271/550] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Aug 30 15:46:37 2022] GET /identity/v3/users/swift => generated 83 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 15:46:38.119402 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740df898-6f97-4df8-9a81-7dfa94f81341 None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:38.146232 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad78d3f-5264-4fae-9c6e-957e967b1389 None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:38.146921 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad78d3f-5264-4fae-9c6e-957e967b1389 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:38.147389 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad78d3f-5264-4fae-9c6e-957e967b1389 None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:38.149185 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 272/551] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 30 15:46:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 15:46:38.154146 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f27a11-bf4e-4910-9a3e-a5e4671e865d None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:38.154640 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f27a11-bf4e-4910-9a3e-a5e4671e865d None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:38.155151 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f27a11-bf4e-4910-9a3e-a5e4671e865d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:38.186789 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-95f27a11-bf4e-4910-9a3e-a5e4671e865d None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:46:38.189424 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-95f27a11-bf4e-4910-9a3e-a5e4671e865d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:46:38.252834 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740df898-6f97-4df8-9a81-7dfa94f81341 None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:38.311655 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbf50cc-86d4-4c2c-acae-01289c60a6f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:38.312621 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbf50cc-86d4-4c2c-acae-01289c60a6f8 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:38.312621 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbf50cc-86d4-4c2c-acae-01289c60a6f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:38.312893 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbf50cc-86d4-4c2c-acae-01289c60a6f8 None admin] PATH_INFO: `/v3/users` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:38.315003 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afbf50cc-86d4-4c2c-acae-01289c60a6f8 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81512) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:46:38.315522 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afbf50cc-86d4-4c2c-acae-01289c60a6f8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:38.318204 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afbf50cc-86d4-4c2c-acae-01289c60a6f8 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:38.331825 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 273/552] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 30 15:46:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 15:46:38.337994 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31c8ac7-1d39-470f-b258-0e35d4bf4660 None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:38.338285 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31c8ac7-1d39-470f-b258-0e35d4bf4660 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:38.338550 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31c8ac7-1d39-470f-b258-0e35d4bf4660 None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:38.340082 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 274/553] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 30 15:46:38 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 15:46:38.355403 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e16ca3-7a9f-47e5-a1cc-54407d05d3ac None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:38.355597 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 280/554] 10.222.0.20 () {56 vars in 1135 bytes} [Tue Aug 30 15:46:38 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:46:38.355776 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e16ca3-7a9f-47e5-a1cc-54407d05d3ac None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:38.355929 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e16ca3-7a9f-47e5-a1cc-54407d05d3ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:38.366202 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbf50cc-86d4-4c2c-acae-01289c60a6f8 None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:38.374734 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-c0e16ca3-7a9f-47e5-a1cc-54407d05d3ac None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:46:38.375970 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-c0e16ca3-7a9f-47e5-a1cc-54407d05d3ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:46:38.383694 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:38.468851 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:38.492229 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 275/555] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 30 15:46:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 15:46:38.515894 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbf50cc-86d4-4c2c-acae-01289c60a6f8 None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:38.538751 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e16ca3-7a9f-47e5-a1cc-54407d05d3ac None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:38.571296 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ba4fad-a354-4c56-9395-a0b807ac7bdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:38.572333 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ba4fad-a354-4c56-9395-a0b807ac7bdd None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:38.572803 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ba4fad-a354-4c56-9395-a0b807ac7bdd None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:38.573308 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ba4fad-a354-4c56-9395-a0b807ac7bdd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:38.650198 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e16ca3-7a9f-47e5-a1cc-54407d05d3ac None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:38.689772 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc77beb-0e78-412a-8bcd-29defc5c3660 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:38.690423 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc77beb-0e78-412a-8bcd-29defc5c3660 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:38.690723 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc77beb-0e78-412a-8bcd-29defc5c3660 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:38.690973 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc77beb-0e78-412a-8bcd-29defc5c3660 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:38.704777 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc77beb-0e78-412a-8bcd-29defc5c3660 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 15:46:38.709690 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ba4fad-a354-4c56-9395-a0b807ac7bdd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:38.710730 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-2cc77beb-0e78-412a-8bcd-29defc5c3660 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 15:46:38.710730 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:46:38.710730 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:46:38.710730 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:46:38.710730 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:46:38.710730 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:46:38.710730 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:46:38.710730 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:46:38.710730 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:46:38.710730 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:46:38.710730 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:46:38.710730 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:46:38.710730 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 15:46:38.710730 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 15:46:38.710730 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 15:46:38.710730 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 15:46:38.711634 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 15:46:38.711634 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 15:46:38.711634 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 15:46:38.711634 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 15:46:38.711634 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:46:38.711634 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:46:38.711634 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:46:38.711634 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:46:38.711634 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:46:38.711634 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:46:38.711634 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:46:38.711634 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Aug 30 15:46:38.711634 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:46:38.711634 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:46:38.711634 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:46:38.711634 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:46:38.712460 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:46:38.712460 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:46:38.712460 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:46:38.712460 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:46:38.712460 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 15:46:38.712460 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 15:46:38.712460 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 15:46:38.712460 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 15:46:38.712460 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 15:46:38.712460 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 15:46:38.712460 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 15:46:38.712460 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Aug 30 15:46:38.713099 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ba4fad-a354-4c56-9395-a0b807ac7bdd None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:38.713178 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 276/556] 10.222.0.20 () {56 vars in 1102 bytes} [Tue Aug 30 15:46:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 15:46:38.713178 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc77beb-0e78-412a-8bcd-29defc5c3660 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:38.717578 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:38.797144 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:38.835033 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc77beb-0e78-412a-8bcd-29defc5c3660 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:38.857859 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 281/557] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Aug 30 15:46:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 487 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 15:46:38.857859 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ba4fad-a354-4c56-9395-a0b807ac7bdd None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:38.874480 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c760548-8809-43fc-adf0-156c784fb76e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:38.875167 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c760548-8809-43fc-adf0-156c784fb76e None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:38.875402 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c760548-8809-43fc-adf0-156c784fb76e None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:38.875667 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c760548-8809-43fc-adf0-156c784fb76e None admin] PATH_INFO: `/v3/roles/service` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:38.886338 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c760548-8809-43fc-adf0-156c784fb76e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 15:46:38.891198 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-3c760548-8809-43fc-adf0-156c784fb76e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 15:46:38.891198 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:46:38.891198 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:46:38.891198 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:46:38.891198 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:46:38.891198 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:46:38.891198 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:46:38.891198 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:46:38.891198 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:46:38.891198 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:46:38.891198 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:46:38.891198 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:46:38.891198 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 15:46:38.891198 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 15:46:38.891198 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 15:46:38.891198 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 15:46:38.892115 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 15:46:38.892115 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 15:46:38.892115 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 15:46:38.892115 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 15:46:38.892115 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:46:38.892115 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:46:38.892115 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:46:38.892115 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:46:38.892115 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:46:38.892115 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:46:38.892115 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:46:38.892115 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Aug 30 15:46:38.892115 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:46:38.892115 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:46:38.892115 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:46:38.892115 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:46:38.892985 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:46:38.892985 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:46:38.892985 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:46:38.892985 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:46:38.892985 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 15:46:38.892985 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 15:46:38.892985 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 15:46:38.892985 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 15:46:38.892985 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 15:46:38.892985 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 15:46:38.892985 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 15:46:38.892985 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Aug 30 15:46:38.892985 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 277/558] 10.222.0.20 () {56 vars in 1102 bytes} [Tue Aug 30 15:46:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 15:46:38.893701 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c760548-8809-43fc-adf0-156c784fb76e None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:38.963930 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ba4fad-a354-4c56-9395-a0b807ac7bdd None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:38.964983 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:39.001364 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cf6f6b-dcbd-4041-89b4-89937a16c61e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:39.001981 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cf6f6b-dcbd-4041-89b4-89937a16c61e None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:39.002270 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cf6f6b-dcbd-4041-89b4-89937a16c61e None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:39.002822 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cf6f6b-dcbd-4041-89b4-89937a16c61e None admin] PATH_INFO: `/v3/roles` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:39.003317 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90cf6f6b-dcbd-4041-89b4-89937a16c61e None admin] RBAC: Adding query filter params (name=service) {{(pid=81512) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:46:39.003503 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90cf6f6b-dcbd-4041-89b4-89937a16c61e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:39.004842 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90cf6f6b-dcbd-4041-89b4-89937a16c61e None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:39.006085 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c760548-8809-43fc-adf0-156c784fb76e None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:39.013319 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 282/559] 10.222.0.20 () {56 vars in 1103 bytes} [Tue Aug 30 15:46:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:46:39.014408 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cf6f6b-dcbd-4041-89b4-89937a16c61e None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:39.039587 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:39.048329 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9521bf7-b33e-4ff5-9e86-3c2f13c104af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:39.048884 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9521bf7-b33e-4ff5-9e86-3c2f13c104af None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:39.049154 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9521bf7-b33e-4ff5-9e86-3c2f13c104af None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:39.049443 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9521bf7-b33e-4ff5-9e86-3c2f13c104af None admin] PATH_INFO: `/v3/roles/service` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:39.059754 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9521bf7-b33e-4ff5-9e86-3c2f13c104af None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 15:46:39.064490 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-f9521bf7-b33e-4ff5-9e86-3c2f13c104af None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 15:46:39.064490 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:46:39.064490 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:46:39.064490 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:46:39.064490 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:46:39.064490 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:46:39.064490 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:46:39.064490 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:46:39.064490 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:46:39.064490 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:46:39.064490 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:46:39.064490 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:46:39.064490 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 15:46:39.064490 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 15:46:39.064490 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 15:46:39.064490 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 15:46:39.065685 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 15:46:39.065685 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 15:46:39.065685 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 15:46:39.065685 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 15:46:39.065685 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:46:39.065685 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:46:39.065685 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:46:39.065685 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:46:39.065685 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:46:39.065685 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:46:39.065685 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:46:39.065685 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Aug 30 15:46:39.065685 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:46:39.065685 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:46:39.065685 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:46:39.065685 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:46:39.066816 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:46:39.066816 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:46:39.066816 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:46:39.066816 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:46:39.066816 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 15:46:39.066816 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 15:46:39.066816 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 15:46:39.066816 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 15:46:39.066816 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 15:46:39.066816 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 15:46:39.066816 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 15:46:39.066816 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Aug 30 15:46:39.070572 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 278/560] 10.222.0.20 () {56 vars in 1102 bytes} [Tue Aug 30 15:46:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 15:46:39.071399 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9521bf7-b33e-4ff5-9e86-3c2f13c104af None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:39.124416 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cf6f6b-dcbd-4041-89b4-89937a16c61e None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:39.158160 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bcd46b-1dfe-4499-aecc-b009f5f31ef2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:39.158787 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bcd46b-1dfe-4499-aecc-b009f5f31ef2 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:39.159058 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bcd46b-1dfe-4499-aecc-b009f5f31ef2 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:39.159318 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bcd46b-1dfe-4499-aecc-b009f5f31ef2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:39.169837 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8bcd46b-1dfe-4499-aecc-b009f5f31ef2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:39.171302 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8bcd46b-1dfe-4499-aecc-b009f5f31ef2 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:39.174922 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9521bf7-b33e-4ff5-9e86-3c2f13c104af None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:39.185430 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application [None req-f8bcd46b-1dfe-4499-aecc-b009f5f31ef2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 15:46:39.185430 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:46:39.185430 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:46:39.185430 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:46:39.185430 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:46:39.185430 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:46:39.185430 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:46:39.185430 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:46:39.185430 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:46:39.185430 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:46:39.185430 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:46:39.185430 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:46:39.185430 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 15:46:39.185430 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 15:46:39.185430 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 15:46:39.185430 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 15:46:39.186547 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:46:39.186547 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:46:39.186547 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:46:39.186547 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:46:39.186547 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:46:39.186547 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:46:39.186547 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:46:39.186547 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application with Lock( Aug 30 15:46:39.186547 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:46:39.186547 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:46:39.186547 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:46:39.186547 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:46:39.186547 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:46:39.186547 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:46:39.186547 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:46:39.186547 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:46:39.186547 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 15:46:39.187632 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 15:46:39.187632 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 15:46:39.187632 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Aug 30 15:46:39.189385 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 283/561] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 30 15:46:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 15:46:39.190449 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bcd46b-1dfe-4499-aecc-b009f5f31ef2 None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:39.198555 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:39.211372 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b72dba-185a-4f15-b86b-1bfcccd8f0f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:39.211984 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b72dba-185a-4f15-b86b-1bfcccd8f0f8 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:39.212261 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b72dba-185a-4f15-b86b-1bfcccd8f0f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:39.212529 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b72dba-185a-4f15-b86b-1bfcccd8f0f8 None admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:39.213199 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b72dba-185a-4f15-b86b-1bfcccd8f0f8 None admin] RBAC: Adding query filter params (name=service) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:46:39.213543 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b72dba-185a-4f15-b86b-1bfcccd8f0f8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:39.214946 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b72dba-185a-4f15-b86b-1bfcccd8f0f8 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:39.222972 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 279/562] 10.222.0.20 () {56 vars in 1103 bytes} [Tue Aug 30 15:46:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:46:39.223729 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b72dba-185a-4f15-b86b-1bfcccd8f0f8 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:39.265442 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:39.294542 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bcd46b-1dfe-4499-aecc-b009f5f31ef2 None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:39.323666 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b72dba-185a-4f15-b86b-1bfcccd8f0f8 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:39.329358 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25166350-c4d5-4cf0-a71b-8506621cdda8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:39.329988 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25166350-c4d5-4cf0-a71b-8506621cdda8 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:39.330264 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25166350-c4d5-4cf0-a71b-8506621cdda8 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:39.330527 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25166350-c4d5-4cf0-a71b-8506621cdda8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:39.341307 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25166350-c4d5-4cf0-a71b-8506621cdda8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:39.342754 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25166350-c4d5-4cf0-a71b-8506621cdda8 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:39.357264 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application [None req-25166350-c4d5-4cf0-a71b-8506621cdda8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 15:46:39.357264 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:46:39.357264 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:46:39.357264 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:46:39.357264 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:46:39.357264 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:46:39.357264 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:46:39.357264 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:46:39.357264 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:46:39.357264 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:46:39.357264 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:46:39.357264 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:46:39.357264 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 15:46:39.357264 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 15:46:39.357264 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 15:46:39.357264 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 15:46:39.358388 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:46:39.358388 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:46:39.358388 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:46:39.358388 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:46:39.358388 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:46:39.358388 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:46:39.358388 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:46:39.358388 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application with Lock( Aug 30 15:46:39.358388 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:46:39.358388 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:46:39.358388 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:46:39.358388 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:46:39.358388 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:46:39.358388 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:46:39.358388 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:46:39.358388 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:46:39.358388 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 15:46:39.364333 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 15:46:39.364333 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 15:46:39.364333 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Aug 30 15:46:39.364518 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1754ecab-9740-48ae-ba6f-2f26b2cc7bd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:39.364518 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1754ecab-9740-48ae-ba6f-2f26b2cc7bd6 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:39.364518 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1754ecab-9740-48ae-ba6f-2f26b2cc7bd6 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:39.364744 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 284/563] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 30 15:46:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 15:46:39.364834 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1754ecab-9740-48ae-ba6f-2f26b2cc7bd6 None admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:39.364834 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1754ecab-9740-48ae-ba6f-2f26b2cc7bd6 None admin] RBAC: Adding query filter params (name=service) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:46:39.364992 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e3ac01-5437-4bbe-ace0-f5047dcac30a None None] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:39.365076 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1754ecab-9740-48ae-ba6f-2f26b2cc7bd6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:39.365156 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e3ac01-5437-4bbe-ace0-f5047dcac30a None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:39.365156 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e3ac01-5437-4bbe-ace0-f5047dcac30a None None] PATH_INFO: `/` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:39.365343 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1754ecab-9740-48ae-ba6f-2f26b2cc7bd6 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:39.365426 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 285/564] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 30 15:46:39 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 15:46:39.365426 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e3ac01-5437-4bbe-ace0-f5047dcac30a None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:39.371912 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 280/565] 10.222.0.20 () {56 vars in 1103 bytes} [Tue Aug 30 15:46:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:46:39.372670 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1754ecab-9740-48ae-ba6f-2f26b2cc7bd6 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:39.420828 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:39.471148 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e3ac01-5437-4bbe-ace0-f5047dcac30a None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:39.477046 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1754ecab-9740-48ae-ba6f-2f26b2cc7bd6 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:39.493239 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:39.504235 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed13d33-b3ce-441b-bfeb-9ad2adfe224f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:39.504912 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed13d33-b3ce-441b-bfeb-9ad2adfe224f None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:39.505189 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed13d33-b3ce-441b-bfeb-9ad2adfe224f None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:39.505490 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed13d33-b3ce-441b-bfeb-9ad2adfe224f None admin] PATH_INFO: `/v3/domains` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:39.506188 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed13d33-b3ce-441b-bfeb-9ad2adfe224f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81512) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:46:39.506490 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed13d33-b3ce-441b-bfeb-9ad2adfe224f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:39.507867 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed13d33-b3ce-441b-bfeb-9ad2adfe224f None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:39.510676 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5056ada4-e572-4f1f-9bc6-d41b16b727f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:39.511303 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5056ada4-e572-4f1f-9bc6-d41b16b727f4 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:39.511583 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5056ada4-e572-4f1f-9bc6-d41b16b727f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:39.511854 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5056ada4-e572-4f1f-9bc6-d41b16b727f4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:39.519662 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 286/566] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Aug 30 15:46:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:46:39.521176 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed13d33-b3ce-441b-bfeb-9ad2adfe224f None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:39.525179 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5056ada4-e572-4f1f-9bc6-d41b16b727f4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:39.526682 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5056ada4-e572-4f1f-9bc6-d41b16b727f4 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:39.542300 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-5056ada4-e572-4f1f-9bc6-d41b16b727f4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 15:46:39.542300 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:46:39.542300 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:46:39.542300 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:46:39.542300 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:46:39.542300 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:46:39.542300 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:46:39.542300 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:46:39.542300 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:46:39.542300 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:46:39.542300 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:46:39.542300 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:46:39.542300 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 15:46:39.542300 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 15:46:39.542300 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 15:46:39.542300 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 15:46:39.543414 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:46:39.543414 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:46:39.543414 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:46:39.543414 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:46:39.543414 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:46:39.543414 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:46:39.543414 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:46:39.543414 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Aug 30 15:46:39.543414 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:46:39.543414 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:46:39.543414 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:46:39.543414 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:46:39.543414 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:46:39.543414 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:46:39.543414 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:46:39.543414 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:46:39.543414 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 15:46:39.544523 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 15:46:39.544523 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 15:46:39.544523 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Aug 30 15:46:39.544523 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 281/567] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 30 15:46:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 15:46:39.545306 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d06460-97f5-4d0b-98ec-190045be63cc None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:39.545585 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d06460-97f5-4d0b-98ec-190045be63cc None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:39.545941 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d06460-97f5-4d0b-98ec-190045be63cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:39.563227 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-00d06460-97f5-4d0b-98ec-190045be63cc None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:46:39.564427 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-00d06460-97f5-4d0b-98ec-190045be63cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:46:39.637596 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed13d33-b3ce-441b-bfeb-9ad2adfe224f None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:39.640156 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 282/568] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 30 15:46:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 15:46:39.641077 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d06460-97f5-4d0b-98ec-190045be63cc None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:39.670175 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:39.677949 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7fc195-b79c-4d92-b722-e867622a2817 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:39.678566 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7fc195-b79c-4d92-b722-e867622a2817 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:39.678832 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7fc195-b79c-4d92-b722-e867622a2817 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:39.679099 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7fc195-b79c-4d92-b722-e867622a2817 None admin] PATH_INFO: `/v3/domains` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:39.679755 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7fc195-b79c-4d92-b722-e867622a2817 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81512) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:46:39.680058 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7fc195-b79c-4d92-b722-e867622a2817 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:39.681481 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7fc195-b79c-4d92-b722-e867622a2817 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:39.693662 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 287/569] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Aug 30 15:46:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:46:39.694776 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7fc195-b79c-4d92-b722-e867622a2817 None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:39.740003 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:39.755704 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d06460-97f5-4d0b-98ec-190045be63cc None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:39.790641 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9733ce87-2687-4932-8371-27dff9b05ed4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:39.791411 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9733ce87-2687-4932-8371-27dff9b05ed4 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:39.791781 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9733ce87-2687-4932-8371-27dff9b05ed4 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:39.792194 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9733ce87-2687-4932-8371-27dff9b05ed4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:39.796947 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7fc195-b79c-4d92-b722-e867622a2817 None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:39.802271 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9733ce87-2687-4932-8371-27dff9b05ed4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:39.803872 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9733ce87-2687-4932-8371-27dff9b05ed4 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:39.819141 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-9733ce87-2687-4932-8371-27dff9b05ed4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 15:46:39.819141 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:46:39.819141 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:46:39.819141 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:46:39.819141 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:46:39.819141 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:46:39.819141 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:46:39.819141 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:46:39.819141 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:46:39.819141 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:46:39.819141 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:46:39.819141 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:46:39.819141 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 15:46:39.819141 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 15:46:39.819141 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 15:46:39.819141 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 15:46:39.820083 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:46:39.820083 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:46:39.820083 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:46:39.820083 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:46:39.820083 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:46:39.820083 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:46:39.820083 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:46:39.820083 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Aug 30 15:46:39.820083 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:46:39.820083 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:46:39.820083 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:46:39.820083 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:46:39.820083 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:46:39.820083 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:46:39.820083 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:46:39.820083 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:46:39.820083 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 15:46:39.821008 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 15:46:39.821008 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 15:46:39.821008 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Aug 30 15:46:39.824938 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 283/570] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 30 15:46:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 15:46:39.825793 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9733ce87-2687-4932-8371-27dff9b05ed4 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:39.833856 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515e024a-a158-416f-a185-5ff8664c4a00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:39.834483 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515e024a-a158-416f-a185-5ff8664c4a00 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:39.834757 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515e024a-a158-416f-a185-5ff8664c4a00 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:39.835019 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515e024a-a158-416f-a185-5ff8664c4a00 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:39.841973 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515e024a-a158-416f-a185-5ff8664c4a00 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:39.843482 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515e024a-a158-416f-a185-5ff8664c4a00 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:39.855394 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application [None req-515e024a-a158-416f-a185-5ff8664c4a00 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 15:46:39.855394 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:46:39.855394 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:46:39.855394 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:46:39.855394 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:46:39.855394 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:46:39.855394 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:46:39.855394 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:46:39.855394 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:46:39.855394 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:46:39.855394 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:46:39.855394 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:46:39.855394 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 15:46:39.855394 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 15:46:39.855394 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 15:46:39.856457 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 15:46:39.856457 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:46:39.856457 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:46:39.856457 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:46:39.856457 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:46:39.856457 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:46:39.856457 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:46:39.856457 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:46:39.856457 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application with Lock( Aug 30 15:46:39.856457 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:46:39.856457 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:46:39.856457 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:46:39.856457 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:46:39.856457 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:46:39.856457 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:46:39.856457 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:46:39.856457 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:46:39.857395 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 15:46:39.857395 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 15:46:39.857395 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 15:46:39.857395 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 15:46:39.857395 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 15:46:39.857395 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 15:46:39.857395 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 15:46:39.857395 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Aug 30 15:46:39.857395 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 288/571] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Aug 30 15:46:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 15:46:39.857395 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515e024a-a158-416f-a185-5ff8664c4a00 None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:39.917078 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:39.937119 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9733ce87-2687-4932-8371-27dff9b05ed4 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:39.962374 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515e024a-a158-416f-a185-5ff8664c4a00 None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:39.964623 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:39.972001 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdeb31c4-dcb1-4c6a-ad75-fe4186317061 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:39.972690 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdeb31c4-dcb1-4c6a-ad75-fe4186317061 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:39.972969 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdeb31c4-dcb1-4c6a-ad75-fe4186317061 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:39.973238 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdeb31c4-dcb1-4c6a-ad75-fe4186317061 None admin] PATH_INFO: `/v3/domains` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:39.973910 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdeb31c4-dcb1-4c6a-ad75-fe4186317061 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:46:39.974211 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdeb31c4-dcb1-4c6a-ad75-fe4186317061 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:39.975600 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdeb31c4-dcb1-4c6a-ad75-fe4186317061 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:39.987649 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 284/572] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Aug 30 15:46:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:46:39.988440 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdeb31c4-dcb1-4c6a-ad75-fe4186317061 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:40.001363 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed6971d-2154-440f-9530-69b11b079518 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:40.001993 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed6971d-2154-440f-9530-69b11b079518 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:40.002270 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed6971d-2154-440f-9530-69b11b079518 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:40.002545 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed6971d-2154-440f-9530-69b11b079518 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:40.010082 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed6971d-2154-440f-9530-69b11b079518 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:40.011518 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed6971d-2154-440f-9530-69b11b079518 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:40.022482 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application [None req-6ed6971d-2154-440f-9530-69b11b079518 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 30 15:46:40.022482 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:46:40.022482 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:46:40.022482 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:46:40.022482 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:46:40.022482 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:46:40.022482 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:46:40.022482 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:46:40.022482 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:46:40.022482 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:46:40.022482 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:46:40.022482 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:46:40.022482 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 15:46:40.022482 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 15:46:40.022482 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 15:46:40.022482 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 15:46:40.023598 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:46:40.023598 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:46:40.023598 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 15:46:40.023598 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 15:46:40.023598 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 15:46:40.023598 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 15:46:40.023598 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:46:40.023598 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:46:40.023598 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:46:40.023598 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:46:40.023598 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:46:40.023598 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application with Lock( Aug 30 15:46:40.023598 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:46:40.023598 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:46:40.023598 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:46:40.023598 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:46:40.023598 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:46:40.023598 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:46:40.024742 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:46:40.024742 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:46:40.024742 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 15:46:40.024742 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 15:46:40.024742 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 15:46:40.024742 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 15:46:40.024742 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 15:46:40.024742 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 15:46:40.024742 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 30 15:46:40.024742 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Aug 30 15:46:40.028220 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 289/573] 10.222.0.20 () {56 vars in 1093 bytes} [Tue Aug 30 15:46:39 2022] GET /identity/v3/users/nova => generated 82 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 15:46:40.028929 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed6971d-2154-440f-9530-69b11b079518 None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:40.087250 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdeb31c4-dcb1-4c6a-ad75-fe4186317061 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:40.120708 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0447552b-762e-408c-bdd6-3a72b283e27e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:40.121676 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0447552b-762e-408c-bdd6-3a72b283e27e None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:40.121953 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0447552b-762e-408c-bdd6-3a72b283e27e None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:40.122219 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0447552b-762e-408c-bdd6-3a72b283e27e None admin] PATH_INFO: `/v3/domains` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:40.122856 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0447552b-762e-408c-bdd6-3a72b283e27e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:46:40.123154 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0447552b-762e-408c-bdd6-3a72b283e27e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:40.124524 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0447552b-762e-408c-bdd6-3a72b283e27e None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:40.127187 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed6971d-2154-440f-9530-69b11b079518 None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:40.132606 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:40.136374 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 285/574] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Aug 30 15:46:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:46:40.137803 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0447552b-762e-408c-bdd6-3a72b283e27e None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:40.161683 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa04b48-797d-43c1-acfb-b9cf3538ad3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:40.162285 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa04b48-797d-43c1-acfb-b9cf3538ad3e None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:40.162563 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa04b48-797d-43c1-acfb-b9cf3538ad3e None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:40.162829 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa04b48-797d-43c1-acfb-b9cf3538ad3e None admin] PATH_INFO: `/v3/projects` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:40.163516 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa04b48-797d-43c1-acfb-b9cf3538ad3e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81512) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:46:40.163817 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa04b48-797d-43c1-acfb-b9cf3538ad3e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:40.165271 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa04b48-797d-43c1-acfb-b9cf3538ad3e None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:40.176930 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 290/575] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Aug 30 15:46:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:46:40.177687 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa04b48-797d-43c1-acfb-b9cf3538ad3e None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:40.207133 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:40.236136 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0447552b-762e-408c-bdd6-3a72b283e27e None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:40.270090 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96836dea-5575-497b-bc4e-1131d06b25d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:40.270716 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96836dea-5575-497b-bc4e-1131d06b25d8 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:40.270989 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96836dea-5575-497b-bc4e-1131d06b25d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:40.271253 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96836dea-5575-497b-bc4e-1131d06b25d8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:40.281340 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96836dea-5575-497b-bc4e-1131d06b25d8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 15:46:40.281815 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa04b48-797d-43c1-acfb-b9cf3538ad3e None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:40.286721 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-96836dea-5575-497b-bc4e-1131d06b25d8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 15:46:40.286721 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:46:40.286721 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:46:40.286721 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:46:40.286721 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:46:40.286721 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:46:40.286721 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:46:40.286721 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:46:40.286721 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:46:40.286721 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:46:40.286721 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:46:40.286721 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:46:40.286721 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 15:46:40.286721 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 15:46:40.286721 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 15:46:40.286721 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 15:46:40.287830 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 15:46:40.287830 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 15:46:40.287830 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 15:46:40.287830 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 15:46:40.287830 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:46:40.287830 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:46:40.287830 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:46:40.287830 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:46:40.287830 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:46:40.287830 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:46:40.287830 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:46:40.287830 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Aug 30 15:46:40.287830 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:46:40.287830 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:46:40.287830 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:46:40.287830 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:46:40.288693 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:46:40.288693 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:46:40.288693 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:46:40.288693 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:46:40.288693 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 15:46:40.288693 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 15:46:40.288693 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 15:46:40.288693 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 15:46:40.288693 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 15:46:40.288693 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 15:46:40.288693 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 15:46:40.288693 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Aug 30 15:46:40.288693 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 286/576] 10.222.0.20 () {56 vars in 1102 bytes} [Tue Aug 30 15:46:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 15:46:40.289431 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96836dea-5575-497b-bc4e-1131d06b25d8 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:40.317023 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2046491-383f-49e7-ba0a-80bbf52d5c40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:40.317712 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2046491-383f-49e7-ba0a-80bbf52d5c40 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:40.318011 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2046491-383f-49e7-ba0a-80bbf52d5c40 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:40.318274 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2046491-383f-49e7-ba0a-80bbf52d5c40 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:40.325577 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2046491-383f-49e7-ba0a-80bbf52d5c40 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:40.327111 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2046491-383f-49e7-ba0a-80bbf52d5c40 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:40.338781 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application [None req-a2046491-383f-49e7-ba0a-80bbf52d5c40 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 30 15:46:40.338781 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:46:40.338781 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:46:40.338781 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:46:40.338781 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:46:40.338781 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:46:40.338781 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:46:40.338781 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:46:40.338781 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:46:40.338781 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:46:40.338781 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:46:40.338781 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:46:40.338781 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 15:46:40.338781 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 15:46:40.338781 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 15:46:40.338781 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 15:46:40.340316 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:46:40.340316 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:46:40.340316 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 15:46:40.340316 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 15:46:40.340316 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 15:46:40.340316 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 15:46:40.340316 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:46:40.340316 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:46:40.340316 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:46:40.340316 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:46:40.340316 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:46:40.340316 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application with Lock( Aug 30 15:46:40.340316 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:46:40.340316 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:46:40.340316 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:46:40.340316 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:46:40.340316 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:46:40.340316 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:46:40.341552 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:46:40.341552 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:46:40.341552 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 15:46:40.341552 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 15:46:40.341552 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 15:46:40.341552 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 15:46:40.341552 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 15:46:40.341552 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 15:46:40.341552 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 30 15:46:40.341552 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Aug 30 15:46:40.341552 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 291/577] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Aug 30 15:46:40 2022] GET /identity/v3/users/cinder => generated 84 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 15:46:40.341552 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2046491-383f-49e7-ba0a-80bbf52d5c40 None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:40.354278 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:40.395180 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96836dea-5575-497b-bc4e-1131d06b25d8 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:40.424953 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:40.428564 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1841f1ed-b266-4ff4-82db-be0866d4f46c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:40.429099 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1841f1ed-b266-4ff4-82db-be0866d4f46c None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:40.429414 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1841f1ed-b266-4ff4-82db-be0866d4f46c None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:40.429687 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1841f1ed-b266-4ff4-82db-be0866d4f46c None admin] PATH_INFO: `/v3/users` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:40.430417 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1841f1ed-b266-4ff4-82db-be0866d4f46c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:46:40.430723 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1841f1ed-b266-4ff4-82db-be0866d4f46c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:40.432174 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1841f1ed-b266-4ff4-82db-be0866d4f46c None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:40.440253 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2046491-383f-49e7-ba0a-80bbf52d5c40 None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:40.449337 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 287/578] 10.222.0.20 () {56 vars in 1133 bytes} [Tue Aug 30 15:46:40 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:46:40.450140 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1841f1ed-b266-4ff4-82db-be0866d4f46c None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:40.480201 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9087a434-012c-4ca2-8785-44d5af1c9ff0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:40.480838 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9087a434-012c-4ca2-8785-44d5af1c9ff0 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:40.481113 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9087a434-012c-4ca2-8785-44d5af1c9ff0 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:40.481418 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9087a434-012c-4ca2-8785-44d5af1c9ff0 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:40.489009 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9087a434-012c-4ca2-8785-44d5af1c9ff0 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:40.490691 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9087a434-012c-4ca2-8785-44d5af1c9ff0 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:40.502315 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application [None req-9087a434-012c-4ca2-8785-44d5af1c9ff0 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 30 15:46:40.502315 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:46:40.502315 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:46:40.502315 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:46:40.502315 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:46:40.502315 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:46:40.502315 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:46:40.502315 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:46:40.502315 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:46:40.502315 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:46:40.502315 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:46:40.502315 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:46:40.502315 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 15:46:40.502315 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 15:46:40.502315 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 15:46:40.502315 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 15:46:40.503318 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:46:40.503318 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:46:40.503318 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 15:46:40.503318 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 15:46:40.503318 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 15:46:40.503318 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 15:46:40.503318 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:46:40.503318 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:46:40.503318 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:46:40.503318 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:46:40.503318 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:46:40.503318 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application with Lock( Aug 30 15:46:40.503318 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:46:40.503318 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:46:40.503318 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:46:40.503318 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:46:40.503318 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:46:40.503318 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:46:40.505026 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:46:40.505026 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:46:40.505026 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 15:46:40.505026 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 15:46:40.505026 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 15:46:40.505026 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 15:46:40.505026 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 15:46:40.505026 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 15:46:40.505026 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 30 15:46:40.505026 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Aug 30 15:46:40.505026 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 292/579] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Aug 30 15:46:40 2022] GET /identity/v3/users/glance => generated 84 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 15:46:40.505026 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9087a434-012c-4ca2-8785-44d5af1c9ff0 None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:40.557967 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1841f1ed-b266-4ff4-82db-be0866d4f46c None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:40.576111 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:40.593297 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa158a6-c657-4e41-9db3-8e41d0a6235d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:40.593910 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa158a6-c657-4e41-9db3-8e41d0a6235d None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:40.594188 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa158a6-c657-4e41-9db3-8e41d0a6235d None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:40.594455 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa158a6-c657-4e41-9db3-8e41d0a6235d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:40.595187 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa158a6-c657-4e41-9db3-8e41d0a6235d None admin] RBAC: Adding query filter params (role.id=57eb8466096346b39319536c6feeb6f7, scope.project.id=854fe71b49224dff9a4b1766a20ab266, user.id=8fdafdc39a594bb4b0aeaf2fbc84c2fb) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:46:40.595490 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa158a6-c657-4e41-9db3-8e41d0a6235d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:40.596943 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa158a6-c657-4e41-9db3-8e41d0a6235d None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:40.601238 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9087a434-012c-4ca2-8785-44d5af1c9ff0 None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:40.602276 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 288/580] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Aug 30 15:46:40 2022] GET /identity/v3/role_assignments?user.id=8fdafdc39a594bb4b0aeaf2fbc84c2fb&scope.project.id=854fe71b49224dff9a4b1766a20ab266&role.id=57eb8466096346b39319536c6feeb6f7 => generated 259 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:46:40.603053 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa158a6-c657-4e41-9db3-8e41d0a6235d None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:40.659166 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a66cf1f-5e92-4a53-b3c7-e231ba126eb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:40.659810 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a66cf1f-5e92-4a53-b3c7-e231ba126eb2 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:40.660084 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a66cf1f-5e92-4a53-b3c7-e231ba126eb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:40.660354 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a66cf1f-5e92-4a53-b3c7-e231ba126eb2 None admin] PATH_INFO: `/v3/roles` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:40.661036 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a66cf1f-5e92-4a53-b3c7-e231ba126eb2 None admin] RBAC: Adding query filter params (name=service) {{(pid=81512) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:46:40.661388 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a66cf1f-5e92-4a53-b3c7-e231ba126eb2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:40.662789 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a66cf1f-5e92-4a53-b3c7-e231ba126eb2 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:40.676308 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 293/581] 10.222.0.20 () {56 vars in 1103 bytes} [Tue Aug 30 15:46:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:46:40.676308 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a66cf1f-5e92-4a53-b3c7-e231ba126eb2 None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:40.682504 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:40.723856 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa158a6-c657-4e41-9db3-8e41d0a6235d None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:40.760891 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368d9bd1-10ef-4f04-9886-50a254cecee1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:40.761539 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368d9bd1-10ef-4f04-9886-50a254cecee1 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:40.761919 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368d9bd1-10ef-4f04-9886-50a254cecee1 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:40.762227 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368d9bd1-10ef-4f04-9886-50a254cecee1 None admin] PATH_INFO: `/v3/users` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:40.762971 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368d9bd1-10ef-4f04-9886-50a254cecee1 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:46:40.763274 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368d9bd1-10ef-4f04-9886-50a254cecee1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:40.764765 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368d9bd1-10ef-4f04-9886-50a254cecee1 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:40.786442 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 289/582] 10.222.0.20 () {56 vars in 1137 bytes} [Tue Aug 30 15:46:40 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:46:40.786442 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368d9bd1-10ef-4f04-9886-50a254cecee1 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:40.786711 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a66cf1f-5e92-4a53-b3c7-e231ba126eb2 None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:40.820713 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd7212f-93bf-41f8-a6c6-4651efdf2deb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:40.821465 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd7212f-93bf-41f8-a6c6-4651efdf2deb None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:40.821819 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd7212f-93bf-41f8-a6c6-4651efdf2deb None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:40.822157 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd7212f-93bf-41f8-a6c6-4651efdf2deb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:40.827649 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:40.835382 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd7212f-93bf-41f8-a6c6-4651efdf2deb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:40.837351 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd7212f-93bf-41f8-a6c6-4651efdf2deb None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:40.857008 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application [None req-7fd7212f-93bf-41f8-a6c6-4651efdf2deb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 15:46:40.857008 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:46:40.857008 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:46:40.857008 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:46:40.857008 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:46:40.857008 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:46:40.857008 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:46:40.857008 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:46:40.857008 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:46:40.857008 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:46:40.857008 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:46:40.857008 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:46:40.857008 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 15:46:40.857008 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 15:46:40.857008 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 15:46:40.857008 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 15:46:40.858031 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:46:40.858031 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:46:40.858031 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:46:40.858031 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:46:40.858031 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:46:40.858031 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:46:40.858031 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:46:40.858031 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application with Lock( Aug 30 15:46:40.858031 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:46:40.858031 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:46:40.858031 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:46:40.858031 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:46:40.858031 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:46:40.858031 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:46:40.858031 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:46:40.858031 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:46:40.858031 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 15:46:40.858997 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 15:46:40.858997 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 15:46:40.858997 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Aug 30 15:46:40.858997 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 294/583] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 30 15:46:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 15:46:40.859652 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd7212f-93bf-41f8-a6c6-4651efdf2deb None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:40.894938 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368d9bd1-10ef-4f04-9886-50a254cecee1 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:40.924043 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:40.931666 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a586aa-484c-496f-8466-7c006da878a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:40.932285 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a586aa-484c-496f-8466-7c006da878a4 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:40.932618 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a586aa-484c-496f-8466-7c006da878a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:40.932873 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a586aa-484c-496f-8466-7c006da878a4 None admin] PATH_INFO: `/v3/users` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:40.933599 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a586aa-484c-496f-8466-7c006da878a4 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:46:40.933902 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a586aa-484c-496f-8466-7c006da878a4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:40.935342 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a586aa-484c-496f-8466-7c006da878a4 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:40.958252 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 290/584] 10.222.0.20 () {56 vars in 1137 bytes} [Tue Aug 30 15:46:40 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:46:40.959203 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a586aa-484c-496f-8466-7c006da878a4 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:40.972308 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd7212f-93bf-41f8-a6c6-4651efdf2deb None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:41.009988 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08350fd7-50e7-4679-b277-3c642785d574 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:41.010634 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08350fd7-50e7-4679-b277-3c642785d574 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:41.010912 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08350fd7-50e7-4679-b277-3c642785d574 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:41.011184 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08350fd7-50e7-4679-b277-3c642785d574 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:41.021940 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08350fd7-50e7-4679-b277-3c642785d574 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:41.023395 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08350fd7-50e7-4679-b277-3c642785d574 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:41.039886 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application [None req-08350fd7-50e7-4679-b277-3c642785d574 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 15:46:41.039886 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:46:41.039886 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:46:41.039886 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:46:41.039886 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:46:41.039886 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:46:41.039886 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:46:41.039886 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:46:41.039886 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:46:41.039886 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:46:41.039886 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:46:41.039886 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:46:41.039886 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 15:46:41.039886 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 15:46:41.039886 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 15:46:41.039886 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 15:46:41.040875 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:46:41.040875 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:46:41.040875 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:46:41.040875 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:46:41.040875 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:46:41.040875 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:46:41.040875 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:46:41.040875 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application with Lock( Aug 30 15:46:41.040875 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:46:41.040875 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:46:41.040875 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:46:41.040875 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:46:41.040875 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:46:41.040875 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:46:41.040875 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:46:41.040875 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:46:41.040875 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 15:46:41.046790 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 15:46:41.046790 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 15:46:41.046790 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Aug 30 15:46:41.046790 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 295/585] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 30 15:46:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 15:46:41.046790 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08350fd7-50e7-4679-b277-3c642785d574 None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:41.074734 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:41.078929 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a586aa-484c-496f-8466-7c006da878a4 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:41.115728 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa855786-fedf-4349-b865-a63dc9ba5aea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:41.116366 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa855786-fedf-4349-b865-a63dc9ba5aea None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:41.116640 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa855786-fedf-4349-b865-a63dc9ba5aea None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:41.116935 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa855786-fedf-4349-b865-a63dc9ba5aea None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:41.127889 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa855786-fedf-4349-b865-a63dc9ba5aea None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:41.129380 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa855786-fedf-4349-b865-a63dc9ba5aea None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:41.144328 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-aa855786-fedf-4349-b865-a63dc9ba5aea None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 15:46:41.144328 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:46:41.144328 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:46:41.144328 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:46:41.144328 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:46:41.144328 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:46:41.144328 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:46:41.144328 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:46:41.144328 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:46:41.144328 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:46:41.144328 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:46:41.144328 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:46:41.144328 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 15:46:41.144328 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 15:46:41.144328 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 15:46:41.144328 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 15:46:41.145315 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:46:41.145315 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:46:41.145315 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:46:41.145315 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:46:41.145315 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:46:41.145315 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:46:41.145315 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:46:41.145315 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Aug 30 15:46:41.145315 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:46:41.145315 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:46:41.145315 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:46:41.145315 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:46:41.145315 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:46:41.145315 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:46:41.145315 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:46:41.145315 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:46:41.145315 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 15:46:41.146218 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 15:46:41.146218 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 15:46:41.146218 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Aug 30 15:46:41.146218 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 291/586] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 30 15:46:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 15:46:41.146218 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa855786-fedf-4349-b865-a63dc9ba5aea None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:41.155740 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08350fd7-50e7-4679-b277-3c642785d574 None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:41.166615 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:41.190335 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9054df-0e5f-4d65-b499-150bbde7f9de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:41.190866 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9054df-0e5f-4d65-b499-150bbde7f9de None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:41.191144 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9054df-0e5f-4d65-b499-150bbde7f9de None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:41.191627 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9054df-0e5f-4d65-b499-150bbde7f9de None admin] PATH_INFO: `/v3/domains` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:41.192114 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9054df-0e5f-4d65-b499-150bbde7f9de None admin] RBAC: Adding query filter params (name=Default) {{(pid=81512) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:46:41.192418 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9054df-0e5f-4d65-b499-150bbde7f9de None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:41.193920 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9054df-0e5f-4d65-b499-150bbde7f9de None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:41.207240 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 296/587] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Aug 30 15:46:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:46:41.208127 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9054df-0e5f-4d65-b499-150bbde7f9de None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:41.260427 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa855786-fedf-4349-b865-a63dc9ba5aea None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:41.294315 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc5d3c7-7c3e-454e-829c-74c2d8c4c07a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:41.294932 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc5d3c7-7c3e-454e-829c-74c2d8c4c07a None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:41.295211 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc5d3c7-7c3e-454e-829c-74c2d8c4c07a None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:41.295477 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc5d3c7-7c3e-454e-829c-74c2d8c4c07a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:41.306744 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc5d3c7-7c3e-454e-829c-74c2d8c4c07a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:41.308206 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc5d3c7-7c3e-454e-829c-74c2d8c4c07a None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:41.311888 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9054df-0e5f-4d65-b499-150bbde7f9de None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:41.321920 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-bbc5d3c7-7c3e-454e-829c-74c2d8c4c07a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 15:46:41.321920 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:46:41.321920 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:46:41.321920 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:46:41.321920 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:46:41.321920 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:46:41.321920 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:46:41.321920 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:46:41.321920 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:46:41.321920 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:46:41.321920 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:46:41.321920 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:46:41.321920 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 15:46:41.321920 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 15:46:41.321920 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 15:46:41.321920 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 15:46:41.323137 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:46:41.323137 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:46:41.323137 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:46:41.323137 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:46:41.323137 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:46:41.323137 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:46:41.323137 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:46:41.323137 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Aug 30 15:46:41.323137 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:46:41.323137 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:46:41.323137 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:46:41.323137 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:46:41.323137 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:46:41.323137 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:46:41.323137 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:46:41.323137 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:46:41.323137 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 15:46:41.324033 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 15:46:41.324033 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 15:46:41.324033 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Aug 30 15:46:41.324033 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 292/588] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 30 15:46:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 15:46:41.324265 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc5d3c7-7c3e-454e-829c-74c2d8c4c07a None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:41.330046 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:41.351678 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4efd71-451f-4671-b609-38c3a903dd52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:41.352282 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4efd71-451f-4671-b609-38c3a903dd52 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:41.352561 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4efd71-451f-4671-b609-38c3a903dd52 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:41.352831 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4efd71-451f-4671-b609-38c3a903dd52 None admin] PATH_INFO: `/v3/domains` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:41.353492 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4efd71-451f-4671-b609-38c3a903dd52 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81512) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:46:41.353794 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4efd71-451f-4671-b609-38c3a903dd52 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:41.355159 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4efd71-451f-4671-b609-38c3a903dd52 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:41.366635 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 297/589] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Aug 30 15:46:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:46:41.367423 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4efd71-451f-4671-b609-38c3a903dd52 None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:41.400778 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:41.443144 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc5d3c7-7c3e-454e-829c-74c2d8c4c07a None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:41.472415 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4efd71-451f-4671-b609-38c3a903dd52 None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:41.476786 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bfb953-9a14-4dd1-882a-86088e6d6f53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:41.477444 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bfb953-9a14-4dd1-882a-86088e6d6f53 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:41.477724 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bfb953-9a14-4dd1-882a-86088e6d6f53 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:41.477994 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bfb953-9a14-4dd1-882a-86088e6d6f53 None admin] PATH_INFO: `/v3/domains` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:41.478640 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53bfb953-9a14-4dd1-882a-86088e6d6f53 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:46:41.478941 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53bfb953-9a14-4dd1-882a-86088e6d6f53 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:41.480310 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53bfb953-9a14-4dd1-882a-86088e6d6f53 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:41.496638 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 293/590] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Aug 30 15:46:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:46:41.496638 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bfb953-9a14-4dd1-882a-86088e6d6f53 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:41.513893 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b721cd-dc3c-4170-8e56-23b47c320fb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:41.514536 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b721cd-dc3c-4170-8e56-23b47c320fb3 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:41.514809 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b721cd-dc3c-4170-8e56-23b47c320fb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:41.515390 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b721cd-dc3c-4170-8e56-23b47c320fb3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:41.522560 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b721cd-dc3c-4170-8e56-23b47c320fb3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:41.524020 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b721cd-dc3c-4170-8e56-23b47c320fb3 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:41.536089 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application [None req-d8b721cd-dc3c-4170-8e56-23b47c320fb3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 15:46:41.536089 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:46:41.536089 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:46:41.536089 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:46:41.536089 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:46:41.536089 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:46:41.536089 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:46:41.536089 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:46:41.536089 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:46:41.536089 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:46:41.536089 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:46:41.536089 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:46:41.536089 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 15:46:41.536089 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 15:46:41.536089 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 15:46:41.537061 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 15:46:41.537061 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:46:41.537061 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:46:41.537061 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:46:41.537061 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:46:41.537061 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:46:41.537061 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:46:41.537061 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:46:41.537061 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application with Lock( Aug 30 15:46:41.537061 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:46:41.537061 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:46:41.537061 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:46:41.537061 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:46:41.537061 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:46:41.537061 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:46:41.537061 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:46:41.537061 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:46:41.538541 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 15:46:41.538541 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 15:46:41.538541 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 15:46:41.538541 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 15:46:41.538541 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 15:46:41.538541 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 15:46:41.538541 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 15:46:41.538541 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Aug 30 15:46:41.538541 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 298/591] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Aug 30 15:46:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 15:46:41.538541 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b721cd-dc3c-4170-8e56-23b47c320fb3 None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:41.567358 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:41.612431 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bfb953-9a14-4dd1-882a-86088e6d6f53 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:41.635756 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:41.647199 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b721cd-dc3c-4170-8e56-23b47c320fb3 None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:41.661860 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f813073-1806-4056-aa4b-3f7e9527be95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:41.662477 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f813073-1806-4056-aa4b-3f7e9527be95 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:41.662754 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f813073-1806-4056-aa4b-3f7e9527be95 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:41.663254 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f813073-1806-4056-aa4b-3f7e9527be95 None admin] PATH_INFO: `/v3/domains` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:41.663666 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f813073-1806-4056-aa4b-3f7e9527be95 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:46:41.663963 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f813073-1806-4056-aa4b-3f7e9527be95 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:41.665351 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f813073-1806-4056-aa4b-3f7e9527be95 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:41.677177 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 294/592] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Aug 30 15:46:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:46:41.677968 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f813073-1806-4056-aa4b-3f7e9527be95 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:41.688640 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58214a96-93fd-48b7-824b-8777023f29c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:41.689311 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58214a96-93fd-48b7-824b-8777023f29c7 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:41.689589 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58214a96-93fd-48b7-824b-8777023f29c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:41.689854 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58214a96-93fd-48b7-824b-8777023f29c7 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:41.698370 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58214a96-93fd-48b7-824b-8777023f29c7 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:41.699756 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58214a96-93fd-48b7-824b-8777023f29c7 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:41.711725 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application [None req-58214a96-93fd-48b7-824b-8777023f29c7 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 30 15:46:41.711725 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:46:41.711725 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:46:41.711725 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:46:41.711725 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:46:41.711725 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:46:41.711725 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:46:41.711725 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:46:41.711725 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:46:41.711725 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:46:41.711725 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:46:41.711725 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:46:41.711725 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 15:46:41.711725 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 15:46:41.711725 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 15:46:41.711725 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 15:46:41.712663 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:46:41.712663 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:46:41.712663 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 15:46:41.712663 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 15:46:41.712663 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 15:46:41.712663 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 15:46:41.712663 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:46:41.712663 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:46:41.712663 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:46:41.712663 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:46:41.712663 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:46:41.712663 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application with Lock( Aug 30 15:46:41.712663 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:46:41.712663 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:46:41.712663 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:46:41.712663 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:46:41.712663 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:46:41.712663 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:46:41.714429 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:46:41.714429 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:46:41.714429 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 15:46:41.714429 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 15:46:41.714429 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 15:46:41.714429 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 15:46:41.714429 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 15:46:41.714429 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 15:46:41.714429 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 30 15:46:41.714429 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Aug 30 15:46:41.714429 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 299/593] 10.222.0.20 () {56 vars in 1102 bytes} [Tue Aug 30 15:46:41 2022] GET /identity/v3/users/neutron => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 15:46:41.714429 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58214a96-93fd-48b7-824b-8777023f29c7 None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:41.788873 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f813073-1806-4056-aa4b-3f7e9527be95 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:41.810274 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:41.818222 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58214a96-93fd-48b7-824b-8777023f29c7 None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:41.822210 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9a2676-d8b8-42f4-b538-3a27825732b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:41.822846 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9a2676-d8b8-42f4-b538-3a27825732b4 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:41.823121 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9a2676-d8b8-42f4-b538-3a27825732b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:41.823390 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9a2676-d8b8-42f4-b538-3a27825732b4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:41.830191 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9a2676-d8b8-42f4-b538-3a27825732b4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:41.831634 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9a2676-d8b8-42f4-b538-3a27825732b4 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:41.842829 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-8a9a2676-d8b8-42f4-b538-3a27825732b4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 15:46:41.842829 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:46:41.842829 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:46:41.842829 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:46:41.842829 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:46:41.842829 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:46:41.842829 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:46:41.842829 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:46:41.842829 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:46:41.842829 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:46:41.842829 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:46:41.842829 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:46:41.842829 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 15:46:41.842829 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 15:46:41.842829 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 15:46:41.843737 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 15:46:41.843737 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:46:41.843737 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:46:41.843737 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:46:41.843737 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:46:41.843737 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:46:41.843737 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:46:41.843737 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:46:41.843737 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Aug 30 15:46:41.843737 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:46:41.843737 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:46:41.843737 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:46:41.843737 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:46:41.843737 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:46:41.843737 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:46:41.843737 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:46:41.843737 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:46:41.844630 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 15:46:41.844630 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 15:46:41.844630 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 15:46:41.844630 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 15:46:41.844630 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 15:46:41.844630 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 15:46:41.844630 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 15:46:41.844630 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Aug 30 15:46:41.844630 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 295/594] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Aug 30 15:46:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 15:46:41.845119 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9a2676-d8b8-42f4-b538-3a27825732b4 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:41.856503 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21829dd-4a7a-4a36-8c87-c1b865649522 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:41.857112 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21829dd-4a7a-4a36-8c87-c1b865649522 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:41.857422 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21829dd-4a7a-4a36-8c87-c1b865649522 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:41.857700 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21829dd-4a7a-4a36-8c87-c1b865649522 None admin] PATH_INFO: `/v3/projects` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:41.858540 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21829dd-4a7a-4a36-8c87-c1b865649522 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81512) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:46:41.858848 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21829dd-4a7a-4a36-8c87-c1b865649522 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:41.860273 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21829dd-4a7a-4a36-8c87-c1b865649522 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:41.877527 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 300/595] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Aug 30 15:46:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:46:41.877527 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21829dd-4a7a-4a36-8c87-c1b865649522 None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:41.887920 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:41.953246 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9a2676-d8b8-42f4-b538-3a27825732b4 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:41.982109 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21829dd-4a7a-4a36-8c87-c1b865649522 None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:41.988050 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999ebd47-ef7b-4396-912f-f1c902f22ea1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:41.988701 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999ebd47-ef7b-4396-912f-f1c902f22ea1 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:41.988977 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999ebd47-ef7b-4396-912f-f1c902f22ea1 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:41.989258 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999ebd47-ef7b-4396-912f-f1c902f22ea1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:41.996386 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999ebd47-ef7b-4396-912f-f1c902f22ea1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:41.997884 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999ebd47-ef7b-4396-912f-f1c902f22ea1 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:42.012113 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-999ebd47-ef7b-4396-912f-f1c902f22ea1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 15:46:42.012113 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:46:42.012113 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:46:42.012113 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:46:42.012113 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:46:42.012113 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:46:42.012113 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:46:42.012113 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:46:42.012113 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:46:42.012113 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:46:42.012113 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:46:42.012113 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:46:42.012113 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 15:46:42.012113 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 15:46:42.012113 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 15:46:42.013086 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 15:46:42.013086 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:46:42.013086 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:46:42.013086 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:46:42.013086 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:46:42.013086 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:46:42.013086 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:46:42.013086 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:46:42.013086 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Aug 30 15:46:42.013086 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:46:42.013086 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:46:42.013086 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:46:42.013086 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:46:42.013086 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:46:42.013086 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:46:42.013086 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:46:42.013086 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:46:42.014007 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 15:46:42.014007 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 15:46:42.014007 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 15:46:42.014007 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 15:46:42.014007 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 15:46:42.014007 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 15:46:42.014007 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 15:46:42.014007 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Aug 30 15:46:42.014007 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 296/596] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Aug 30 15:46:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 15:46:42.015048 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999ebd47-ef7b-4396-912f-f1c902f22ea1 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:42.024142 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc4156a-a212-4ec0-8e8f-9d95afab46cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:42.024773 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc4156a-a212-4ec0-8e8f-9d95afab46cd None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:42.025039 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc4156a-a212-4ec0-8e8f-9d95afab46cd None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:42.025343 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc4156a-a212-4ec0-8e8f-9d95afab46cd None admin] PATH_INFO: `/v3/users` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:42.026083 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc4156a-a212-4ec0-8e8f-9d95afab46cd None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81512) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:46:42.026390 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc4156a-a212-4ec0-8e8f-9d95afab46cd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:42.027840 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc4156a-a212-4ec0-8e8f-9d95afab46cd None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:42.050084 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 301/597] 10.222.0.20 () {56 vars in 1139 bytes} [Tue Aug 30 15:46:41 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:46:42.050084 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc4156a-a212-4ec0-8e8f-9d95afab46cd None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:42.064189 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:42.113766 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:42.125457 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999ebd47-ef7b-4396-912f-f1c902f22ea1 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:42.147985 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc4156a-a212-4ec0-8e8f-9d95afab46cd None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:42.161502 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8920ef5c-8e5b-435f-8dbb-c3caab2819df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:42.162129 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8920ef5c-8e5b-435f-8dbb-c3caab2819df None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:42.162402 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8920ef5c-8e5b-435f-8dbb-c3caab2819df None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:42.162724 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8920ef5c-8e5b-435f-8dbb-c3caab2819df None admin] PATH_INFO: `/v3/projects` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:42.163704 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8920ef5c-8e5b-435f-8dbb-c3caab2819df None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:46:42.163704 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8920ef5c-8e5b-435f-8dbb-c3caab2819df None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:42.165141 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8920ef5c-8e5b-435f-8dbb-c3caab2819df None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:42.182419 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 297/598] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Aug 30 15:46:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:46:42.182419 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8920ef5c-8e5b-435f-8dbb-c3caab2819df None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:42.189157 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57319cee-0dad-4da1-824b-6ffbea72ee35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:42.189853 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57319cee-0dad-4da1-824b-6ffbea72ee35 None admin] REQUEST_METHOD: `PUT` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:42.190125 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57319cee-0dad-4da1-824b-6ffbea72ee35 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:42.190388 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57319cee-0dad-4da1-824b-6ffbea72ee35 None admin] PATH_INFO: `/v3/projects/854fe71b49224dff9a4b1766a20ab266/users/7a1e756ae9be4a7bac1e81acf07e8f3b/roles/57eb8466096346b39319536c6feeb6f7` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:42.206020 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57319cee-0dad-4da1-824b-6ffbea72ee35 None admin] RBAC: Authorizing `identity:create_grant(project_id=854fe71b49224dff9a4b1766a20ab266, user_id=7a1e756ae9be4a7bac1e81acf07e8f3b, role_id=57eb8466096346b39319536c6feeb6f7)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:42.207378 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57319cee-0dad-4da1-824b-6ffbea72ee35 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:42.222115 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 302/599] 10.222.0.20 () {58 vars in 1439 bytes} [Tue Aug 30 15:46:42 2022] PUT /identity/v3/projects/854fe71b49224dff9a4b1766a20ab266/users/7a1e756ae9be4a7bac1e81acf07e8f3b/roles/57eb8466096346b39319536c6feeb6f7 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:46:42.222992 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57319cee-0dad-4da1-824b-6ffbea72ee35 None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:42.293703 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8920ef5c-8e5b-435f-8dbb-c3caab2819df None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:42.296278 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:42.336347 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1a1a6c-c92e-47df-a6f6-e0cf9f6d5491 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:42.336965 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1a1a6c-c92e-47df-a6f6-e0cf9f6d5491 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:42.337255 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1a1a6c-c92e-47df-a6f6-e0cf9f6d5491 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:42.337538 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1a1a6c-c92e-47df-a6f6-e0cf9f6d5491 None admin] PATH_INFO: `/v3/projects` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:42.338239 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1a1a6c-c92e-47df-a6f6-e0cf9f6d5491 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:46:42.338586 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1a1a6c-c92e-47df-a6f6-e0cf9f6d5491 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:42.340032 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1a1a6c-c92e-47df-a6f6-e0cf9f6d5491 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:42.357362 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 298/600] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Aug 30 15:46:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:46:42.357362 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1a1a6c-c92e-47df-a6f6-e0cf9f6d5491 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:42.379789 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57319cee-0dad-4da1-824b-6ffbea72ee35 None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:42.399767 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:42.422044 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6bf150-2560-40d2-bd44-00994a14c834 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:42.422682 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6bf150-2560-40d2-bd44-00994a14c834 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:42.422956 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6bf150-2560-40d2-bd44-00994a14c834 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:42.423221 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6bf150-2560-40d2-bd44-00994a14c834 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:42.434324 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6bf150-2560-40d2-bd44-00994a14c834 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:42.435287 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6bf150-2560-40d2-bd44-00994a14c834 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:42.452878 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application [None req-5d6bf150-2560-40d2-bd44-00994a14c834 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 15:46:42.452878 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:46:42.452878 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:46:42.452878 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:46:42.452878 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:46:42.452878 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:46:42.452878 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:46:42.452878 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:46:42.452878 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:46:42.452878 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:46:42.452878 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:46:42.452878 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:46:42.452878 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 15:46:42.452878 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 15:46:42.452878 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 15:46:42.452878 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 15:46:42.453920 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:46:42.453920 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:46:42.453920 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:46:42.453920 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:46:42.453920 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:46:42.453920 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:46:42.453920 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:46:42.453920 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application with Lock( Aug 30 15:46:42.453920 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:46:42.453920 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:46:42.453920 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:46:42.453920 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:46:42.453920 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:46:42.453920 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:46:42.453920 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:46:42.453920 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:46:42.453920 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 15:46:42.454846 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 15:46:42.454846 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 15:46:42.454846 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Aug 30 15:46:42.455027 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 303/601] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 30 15:46:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 15:46:42.456705 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6bf150-2560-40d2-bd44-00994a14c834 None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:42.476075 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1a1a6c-c92e-47df-a6f6-e0cf9f6d5491 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:42.512141 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7a465e-7bce-4938-8786-7635a288f6a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:42.513268 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7a465e-7bce-4938-8786-7635a288f6a9 None admin] REQUEST_METHOD: `PUT` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:42.513539 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7a465e-7bce-4938-8786-7635a288f6a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:42.513810 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7a465e-7bce-4938-8786-7635a288f6a9 None admin] PATH_INFO: `/v3/projects/854fe71b49224dff9a4b1766a20ab266/users/0100a04821804993866ea20f159663f9/roles/57eb8466096346b39319536c6feeb6f7` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:42.532820 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7a465e-7bce-4938-8786-7635a288f6a9 None admin] RBAC: Authorizing `identity:create_grant(project_id=854fe71b49224dff9a4b1766a20ab266, user_id=0100a04821804993866ea20f159663f9, role_id=57eb8466096346b39319536c6feeb6f7)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:42.534523 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7a465e-7bce-4938-8786-7635a288f6a9 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:42.561255 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 299/602] 10.222.0.20 () {58 vars in 1439 bytes} [Tue Aug 30 15:46:42 2022] PUT /identity/v3/projects/854fe71b49224dff9a4b1766a20ab266/users/0100a04821804993866ea20f159663f9/roles/57eb8466096346b39319536c6feeb6f7 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:46:42.561255 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7a465e-7bce-4938-8786-7635a288f6a9 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:42.584643 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6bf150-2560-40d2-bd44-00994a14c834 None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:42.585573 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:42.632435 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca427494-f86b-4f96-949e-3a53d59f8ff3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:42.632435 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca427494-f86b-4f96-949e-3a53d59f8ff3 None admin] REQUEST_METHOD: `PUT` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:42.632435 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca427494-f86b-4f96-949e-3a53d59f8ff3 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:42.632435 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca427494-f86b-4f96-949e-3a53d59f8ff3 None admin] PATH_INFO: `/v3/projects/854fe71b49224dff9a4b1766a20ab266/users/b70e06a7d06d4199ab4fc36342ee1d68/roles/57eb8466096346b39319536c6feeb6f7` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:42.659634 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca427494-f86b-4f96-949e-3a53d59f8ff3 None admin] RBAC: Authorizing `identity:create_grant(project_id=854fe71b49224dff9a4b1766a20ab266, user_id=b70e06a7d06d4199ab4fc36342ee1d68, role_id=57eb8466096346b39319536c6feeb6f7)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:42.661269 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca427494-f86b-4f96-949e-3a53d59f8ff3 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:42.683947 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 304/603] 10.222.0.20 () {58 vars in 1439 bytes} [Tue Aug 30 15:46:42 2022] PUT /identity/v3/projects/854fe71b49224dff9a4b1766a20ab266/users/b70e06a7d06d4199ab4fc36342ee1d68/roles/57eb8466096346b39319536c6feeb6f7 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:46:42.685733 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca26c390-f1e4-416e-a2fe-8ecb01166e32 None None] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:42.686652 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca26c390-f1e4-416e-a2fe-8ecb01166e32 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:42.687044 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca26c390-f1e4-416e-a2fe-8ecb01166e32 None None] PATH_INFO: `/` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:42.689384 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 305/604] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 30 15:46:42 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 15:46:42.696253 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bfce0c-807a-4664-b433-50df740ce08a None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:42.696561 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bfce0c-807a-4664-b433-50df740ce08a None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:42.696853 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bfce0c-807a-4664-b433-50df740ce08a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:42.716172 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7a465e-7bce-4938-8786-7635a288f6a9 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:42.719168 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-c5bfce0c-807a-4664-b433-50df740ce08a None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:46:42.720421 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-c5bfce0c-807a-4664-b433-50df740ce08a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:46:42.748735 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:42.758464 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400e8d8c-9f8a-4a14-936a-3878b34bff98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:42.759093 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400e8d8c-9f8a-4a14-936a-3878b34bff98 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:42.759375 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400e8d8c-9f8a-4a14-936a-3878b34bff98 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:42.759697 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400e8d8c-9f8a-4a14-936a-3878b34bff98 None admin] PATH_INFO: `/v3/domains` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:42.760390 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400e8d8c-9f8a-4a14-936a-3878b34bff98 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:46:42.760696 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400e8d8c-9f8a-4a14-936a-3878b34bff98 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:42.762322 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400e8d8c-9f8a-4a14-936a-3878b34bff98 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:42.778232 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 300/605] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Aug 30 15:46:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:46:42.786968 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400e8d8c-9f8a-4a14-936a-3878b34bff98 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:42.828621 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 306/606] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 30 15:46:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 15:46:42.865598 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:42.898426 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400e8d8c-9f8a-4a14-936a-3878b34bff98 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:42.935024 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9790ae-a67a-4e39-93be-5bf04e2a2b22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:42.935691 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9790ae-a67a-4e39-93be-5bf04e2a2b22 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:42.935961 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9790ae-a67a-4e39-93be-5bf04e2a2b22 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:42.936223 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9790ae-a67a-4e39-93be-5bf04e2a2b22 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:42.943429 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9790ae-a67a-4e39-93be-5bf04e2a2b22 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:42.944901 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9790ae-a67a-4e39-93be-5bf04e2a2b22 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:42.957308 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-3b9790ae-a67a-4e39-93be-5bf04e2a2b22 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 15:46:42.957308 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:46:42.957308 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:46:42.957308 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:46:42.957308 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:46:42.957308 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:46:42.957308 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:46:42.957308 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:46:42.957308 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:46:42.957308 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:46:42.957308 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:46:42.957308 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:46:42.957308 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 15:46:42.957308 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 15:46:42.957308 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 15:46:42.958237 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 15:46:42.958237 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:46:42.958237 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:46:42.958237 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:46:42.958237 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:46:42.958237 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:46:42.958237 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:46:42.958237 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:46:42.958237 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Aug 30 15:46:42.958237 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:46:42.958237 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:46:42.958237 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:46:42.958237 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:46:42.958237 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:46:42.958237 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:46:42.958237 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:46:42.958237 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:46:42.959584 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 15:46:42.959584 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 15:46:42.959584 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 15:46:42.959584 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 15:46:42.959584 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 15:46:42.959584 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 15:46:42.959584 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 15:46:42.959584 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Aug 30 15:46:42.959584 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 301/607] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Aug 30 15:46:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 15:46:42.964817 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bfce0c-807a-4664-b433-50df740ce08a None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:43.071155 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bfce0c-807a-4664-b433-50df740ce08a None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:43.121918 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfdd3aa-4c7d-49fc-a081-ac93c069dbe7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:43.122530 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfdd3aa-4c7d-49fc-a081-ac93c069dbe7 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:43.122805 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfdd3aa-4c7d-49fc-a081-ac93c069dbe7 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:43.123070 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfdd3aa-4c7d-49fc-a081-ac93c069dbe7 None admin] PATH_INFO: `/v3/projects` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:43.123752 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbfdd3aa-4c7d-49fc-a081-ac93c069dbe7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81512) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:46:43.124077 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbfdd3aa-4c7d-49fc-a081-ac93c069dbe7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:43.126075 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9790ae-a67a-4e39-93be-5bf04e2a2b22 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:43.127012 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbfdd3aa-4c7d-49fc-a081-ac93c069dbe7 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:43.148134 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 307/608] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Aug 30 15:46:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:46:43.156355 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfdd3aa-4c7d-49fc-a081-ac93c069dbe7 None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:43.177564 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:43.262245 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9790ae-a67a-4e39-93be-5bf04e2a2b22 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:43.296448 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:43.303673 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90066ef9-6620-4ead-8f52-312dc5804104 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:43.304312 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90066ef9-6620-4ead-8f52-312dc5804104 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:43.304591 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90066ef9-6620-4ead-8f52-312dc5804104 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:43.304863 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90066ef9-6620-4ead-8f52-312dc5804104 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:43.322165 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90066ef9-6620-4ead-8f52-312dc5804104 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 15:46:43.327898 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-90066ef9-6620-4ead-8f52-312dc5804104 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 15:46:43.327898 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:46:43.327898 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:46:43.327898 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:46:43.327898 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:46:43.327898 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:46:43.327898 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:46:43.327898 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:46:43.327898 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:46:43.327898 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:46:43.327898 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:46:43.327898 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:46:43.327898 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 15:46:43.327898 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 15:46:43.327898 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 15:46:43.327898 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 15:46:43.328819 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 15:46:43.328819 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 15:46:43.328819 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 15:46:43.328819 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 15:46:43.328819 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:46:43.328819 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:46:43.328819 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:46:43.328819 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:46:43.328819 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:46:43.328819 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:46:43.328819 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:46:43.328819 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Aug 30 15:46:43.328819 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:46:43.328819 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:46:43.328819 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:46:43.328819 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:46:43.330689 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:46:43.330689 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:46:43.330689 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:46:43.330689 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:46:43.330689 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 15:46:43.330689 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 15:46:43.330689 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 15:46:43.330689 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 15:46:43.330689 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 15:46:43.330689 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 15:46:43.330689 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 15:46:43.330689 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Aug 30 15:46:43.330689 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 302/609] 10.222.0.20 () {56 vars in 1102 bytes} [Tue Aug 30 15:46:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 15:46:43.338319 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90066ef9-6620-4ead-8f52-312dc5804104 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:43.347689 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfdd3aa-4c7d-49fc-a081-ac93c069dbe7 None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:43.381940 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708388e2-1c27-4da1-8e11-74c8c8803a03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:43.383441 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708388e2-1c27-4da1-8e11-74c8c8803a03 None admin] REQUEST_METHOD: `PUT` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:43.383914 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708388e2-1c27-4da1-8e11-74c8c8803a03 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:43.384375 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708388e2-1c27-4da1-8e11-74c8c8803a03 None admin] PATH_INFO: `/v3/projects/854fe71b49224dff9a4b1766a20ab266/users/65f4157105f3488f807b7475008ab722/roles/57eb8466096346b39319536c6feeb6f7` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:43.399954 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708388e2-1c27-4da1-8e11-74c8c8803a03 None admin] RBAC: Authorizing `identity:create_grant(project_id=854fe71b49224dff9a4b1766a20ab266, user_id=65f4157105f3488f807b7475008ab722, role_id=57eb8466096346b39319536c6feeb6f7)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:43.401882 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708388e2-1c27-4da1-8e11-74c8c8803a03 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:43.423357 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 308/610] 10.222.0.20 () {58 vars in 1439 bytes} [Tue Aug 30 15:46:43 2022] PUT /identity/v3/projects/854fe71b49224dff9a4b1766a20ab266/users/65f4157105f3488f807b7475008ab722/roles/57eb8466096346b39319536c6feeb6f7 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:46:43.505601 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90066ef9-6620-4ead-8f52-312dc5804104 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:43.545947 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd55e87a-1e23-4848-af08-af3c250badaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:43.546988 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd55e87a-1e23-4848-af08-af3c250badaa None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:43.547489 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd55e87a-1e23-4848-af08-af3c250badaa None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:43.547952 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd55e87a-1e23-4848-af08-af3c250badaa None admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:43.548858 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd55e87a-1e23-4848-af08-af3c250badaa None admin] RBAC: Adding query filter params (name=service) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:46:43.549406 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd55e87a-1e23-4848-af08-af3c250badaa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:43.551229 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd55e87a-1e23-4848-af08-af3c250badaa None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:43.565561 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 303/611] 10.222.0.20 () {56 vars in 1103 bytes} [Tue Aug 30 15:46:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:46:43.571344 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708388e2-1c27-4da1-8e11-74c8c8803a03 None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:43.662538 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:43.707530 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708388e2-1c27-4da1-8e11-74c8c8803a03 None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:43.741062 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3766526b-ae0a-464f-937d-8fa1f36f0788 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:43.741732 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3766526b-ae0a-464f-937d-8fa1f36f0788 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:43.742014 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3766526b-ae0a-464f-937d-8fa1f36f0788 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:43.742287 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3766526b-ae0a-464f-937d-8fa1f36f0788 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:43.753859 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3766526b-ae0a-464f-937d-8fa1f36f0788 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:43.755320 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3766526b-ae0a-464f-937d-8fa1f36f0788 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:43.770172 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application [None req-3766526b-ae0a-464f-937d-8fa1f36f0788 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 15:46:43.770172 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:46:43.770172 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:46:43.770172 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:46:43.770172 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:46:43.770172 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:46:43.770172 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:46:43.770172 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:46:43.770172 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:46:43.770172 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:46:43.770172 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:46:43.770172 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:46:43.770172 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 15:46:43.770172 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 15:46:43.770172 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 15:46:43.770172 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 15:46:43.771113 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:46:43.771113 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:46:43.771113 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:46:43.771113 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:46:43.771113 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:46:43.771113 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:46:43.771113 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:46:43.771113 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application with Lock( Aug 30 15:46:43.771113 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:46:43.771113 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:46:43.771113 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:46:43.771113 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:46:43.771113 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:46:43.771113 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:46:43.771113 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:46:43.771113 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:46:43.771113 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 15:46:43.772152 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 15:46:43.772152 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 15:46:43.772152 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Aug 30 15:46:43.772152 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 309/612] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 30 15:46:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 15:46:43.776994 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd55e87a-1e23-4848-af08-af3c250badaa None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:43.867111 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:43.928831 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd55e87a-1e23-4848-af08-af3c250badaa None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:43.978818 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379a6652-7b27-486f-827e-8f1e32e06da1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:43.979445 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379a6652-7b27-486f-827e-8f1e32e06da1 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:43.979718 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379a6652-7b27-486f-827e-8f1e32e06da1 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:43.979986 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379a6652-7b27-486f-827e-8f1e32e06da1 None admin] PATH_INFO: `/v3/domains` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:43.980636 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379a6652-7b27-486f-827e-8f1e32e06da1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:46:43.980934 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379a6652-7b27-486f-827e-8f1e32e06da1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:43.982360 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379a6652-7b27-486f-827e-8f1e32e06da1 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:44.002113 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 304/613] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Aug 30 15:46:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:46:44.007682 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3766526b-ae0a-464f-937d-8fa1f36f0788 None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:44.172994 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3766526b-ae0a-464f-937d-8fa1f36f0788 None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:44.191993 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:44.215249 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a9a836-19d9-442e-b599-e5e4a148cd98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:44.216187 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a9a836-19d9-442e-b599-e5e4a148cd98 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:44.216672 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a9a836-19d9-442e-b599-e5e4a148cd98 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:44.217119 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a9a836-19d9-442e-b599-e5e4a148cd98 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:44.226061 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a9a836-19d9-442e-b599-e5e4a148cd98 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:44.227730 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a9a836-19d9-442e-b599-e5e4a148cd98 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:44.242217 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application [None req-d3a9a836-19d9-442e-b599-e5e4a148cd98 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 30 15:46:44.242217 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:46:44.242217 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:46:44.242217 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:46:44.242217 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:46:44.242217 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:46:44.242217 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:46:44.242217 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:46:44.242217 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:46:44.242217 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:46:44.242217 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:46:44.242217 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:46:44.242217 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 15:46:44.242217 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 15:46:44.242217 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 15:46:44.242217 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 15:46:44.243237 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:46:44.243237 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:46:44.243237 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 15:46:44.243237 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 15:46:44.243237 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 15:46:44.243237 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 15:46:44.243237 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:46:44.243237 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:46:44.243237 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:46:44.243237 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:46:44.243237 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:46:44.243237 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application with Lock( Aug 30 15:46:44.243237 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:46:44.243237 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:46:44.243237 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:46:44.243237 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:46:44.243237 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:46:44.243237 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:46:44.244180 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:46:44.244180 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:46:44.244180 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 15:46:44.244180 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 15:46:44.244180 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 15:46:44.244180 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 15:46:44.244180 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 15:46:44.244180 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 15:46:44.244180 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 30 15:46:44.244180 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Aug 30 15:46:44.247137 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 310/614] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Aug 30 15:46:44 2022] GET /identity/v3/users/swift => generated 83 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 15:46:44.252612 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379a6652-7b27-486f-827e-8f1e32e06da1 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:44.396807 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379a6652-7b27-486f-827e-8f1e32e06da1 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:44.418115 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:44.446908 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490c36a5-34e7-47be-a486-5736c9f9fd30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:44.447448 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490c36a5-34e7-47be-a486-5736c9f9fd30 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:44.447722 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490c36a5-34e7-47be-a486-5736c9f9fd30 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:44.447986 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490c36a5-34e7-47be-a486-5736c9f9fd30 None admin] PATH_INFO: `/v3/users` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:44.448730 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490c36a5-34e7-47be-a486-5736c9f9fd30 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:46:44.449031 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490c36a5-34e7-47be-a486-5736c9f9fd30 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:44.450514 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490c36a5-34e7-47be-a486-5736c9f9fd30 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:44.475166 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 305/615] 10.222.0.20 () {56 vars in 1135 bytes} [Tue Aug 30 15:46:44 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:46:44.480691 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a9a836-19d9-442e-b599-e5e4a148cd98 None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:44.635900 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a9a836-19d9-442e-b599-e5e4a148cd98 None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:44.683436 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab4f5f4-9594-4d37-819e-91819373616e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:44.684073 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab4f5f4-9594-4d37-819e-91819373616e None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:44.684349 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab4f5f4-9594-4d37-819e-91819373616e None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:44.684619 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab4f5f4-9594-4d37-819e-91819373616e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:44.693088 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab965bbd-b8eb-4c7a-a3e5-a3bcddedafa3 None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:44.693396 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab965bbd-b8eb-4c7a-a3e5-a3bcddedafa3 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:44.693672 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab965bbd-b8eb-4c7a-a3e5-a3bcddedafa3 None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:44.695256 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 306/616] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 30 15:46:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 15:46:44.695842 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab4f5f4-9594-4d37-819e-91819373616e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:44.697511 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab4f5f4-9594-4d37-819e-91819373616e None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:44.700113 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986ee05d-e0eb-42f8-b797-7cd9c2d246a5 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:44.700391 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986ee05d-e0eb-42f8-b797-7cd9c2d246a5 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:44.700662 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986ee05d-e0eb-42f8-b797-7cd9c2d246a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:44.712413 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application [None req-7ab4f5f4-9594-4d37-819e-91819373616e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 15:46:44.712413 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:46:44.712413 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:46:44.712413 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:46:44.712413 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:46:44.712413 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:46:44.712413 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:46:44.712413 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:46:44.712413 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:46:44.712413 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:46:44.712413 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:46:44.712413 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:46:44.712413 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 15:46:44.712413 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 15:46:44.712413 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 15:46:44.712413 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 15:46:44.713929 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:46:44.713929 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:46:44.713929 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:46:44.713929 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:46:44.713929 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:46:44.713929 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:46:44.713929 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:46:44.713929 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application with Lock( Aug 30 15:46:44.713929 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:46:44.713929 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:46:44.713929 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:46:44.713929 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:46:44.713929 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:46:44.713929 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:46:44.713929 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:46:44.713929 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:46:44.713929 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 15:46:44.714822 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 15:46:44.714822 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 15:46:44.714822 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Aug 30 15:46:44.714822 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 311/617] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 30 15:46:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 15:46:44.719271 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab4f5f4-9594-4d37-819e-91819373616e None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:44.732784 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-986ee05d-e0eb-42f8-b797-7cd9c2d246a5 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:46:44.734108 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-986ee05d-e0eb-42f8-b797-7cd9c2d246a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:46:44.762704 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:44.858715 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 307/618] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 30 15:46:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 15:46:44.885306 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab4f5f4-9594-4d37-819e-91819373616e None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:44.933516 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df97fb9e-d0dd-4b31-8711-a226d7ee80f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:44.934148 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df97fb9e-d0dd-4b31-8711-a226d7ee80f0 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:44.934428 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df97fb9e-d0dd-4b31-8711-a226d7ee80f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:44.934700 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df97fb9e-d0dd-4b31-8711-a226d7ee80f0 None admin] PATH_INFO: `/v3/domains` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:44.935367 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df97fb9e-d0dd-4b31-8711-a226d7ee80f0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81512) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:46:44.935671 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df97fb9e-d0dd-4b31-8711-a226d7ee80f0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:44.937254 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df97fb9e-d0dd-4b31-8711-a226d7ee80f0 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:44.953105 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 312/619] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Aug 30 15:46:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:46:44.958728 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986ee05d-e0eb-42f8-b797-7cd9c2d246a5 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:45.010614 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbfdf15-683e-4420-9d21-70d61184df7d None None] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:45.010928 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbfdf15-683e-4420-9d21-70d61184df7d None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:45.011149 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbfdf15-683e-4420-9d21-70d61184df7d None None] PATH_INFO: `/` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:45.013742 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 313/620] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 30 15:46:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 15:46:45.017808 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea32aca8-d965-4065-a494-04acf04b7ef8 None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:45.018087 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea32aca8-d965-4065-a494-04acf04b7ef8 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:45.018357 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea32aca8-d965-4065-a494-04acf04b7ef8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:45.036383 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-ea32aca8-d965-4065-a494-04acf04b7ef8 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:46:45.037700 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-ea32aca8-d965-4065-a494-04acf04b7ef8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:46:45.090031 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986ee05d-e0eb-42f8-b797-7cd9c2d246a5 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:45.093522 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:45.123234 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 314/621] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 30 15:46:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 15:46:45.141929 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03095128-7fec-4fae-9f48-d148d13b7a3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:45.142646 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03095128-7fec-4fae-9f48-d148d13b7a3d None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:45.142934 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03095128-7fec-4fae-9f48-d148d13b7a3d None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:45.143209 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03095128-7fec-4fae-9f48-d148d13b7a3d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:45.150277 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03095128-7fec-4fae-9f48-d148d13b7a3d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:45.151736 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03095128-7fec-4fae-9f48-d148d13b7a3d None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:45.167030 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-03095128-7fec-4fae-9f48-d148d13b7a3d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 15:46:45.167030 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:46:45.167030 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:46:45.167030 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:46:45.167030 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:46:45.167030 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:46:45.167030 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:46:45.167030 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:46:45.167030 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:46:45.167030 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:46:45.167030 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:46:45.167030 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:46:45.167030 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 15:46:45.167030 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 15:46:45.167030 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 15:46:45.167957 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 15:46:45.167957 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:46:45.167957 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:46:45.167957 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:46:45.167957 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:46:45.167957 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:46:45.167957 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:46:45.167957 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:46:45.167957 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Aug 30 15:46:45.167957 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:46:45.167957 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:46:45.167957 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:46:45.167957 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:46:45.167957 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:46:45.167957 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:46:45.167957 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:46:45.167957 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:46:45.168844 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 15:46:45.168844 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 15:46:45.168844 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 15:46:45.168844 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 15:46:45.168844 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 15:46:45.168844 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 15:46:45.168844 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 15:46:45.168844 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Aug 30 15:46:45.168844 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 308/622] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Aug 30 15:46:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 15:46:45.175612 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea32aca8-d965-4065-a494-04acf04b7ef8 None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:45.192320 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:45.281346 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba15d525-bd7a-4c6a-9a7a-95c8ff2e5e96 None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:45.281520 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba15d525-bd7a-4c6a-9a7a-95c8ff2e5e96 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:45.281738 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea32aca8-d965-4065-a494-04acf04b7ef8 None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:45.281839 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba15d525-bd7a-4c6a-9a7a-95c8ff2e5e96 None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:45.283400 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 309/623] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 30 15:46:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 15:46:45.288190 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4935ff85-1d06-43bd-9327-50590d810bd5 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:45.288525 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4935ff85-1d06-43bd-9327-50590d810bd5 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:45.288802 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4935ff85-1d06-43bd-9327-50590d810bd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:45.307422 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-4935ff85-1d06-43bd-9327-50590d810bd5 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:46:45.308599 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-4935ff85-1d06-43bd-9327-50590d810bd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:46:45.317732 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988b5dff-2813-4261-b300-05ad75a1cacb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:45.318349 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988b5dff-2813-4261-b300-05ad75a1cacb None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:45.318631 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988b5dff-2813-4261-b300-05ad75a1cacb None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:45.318908 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988b5dff-2813-4261-b300-05ad75a1cacb None admin] PATH_INFO: `/v3/projects` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:45.319598 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988b5dff-2813-4261-b300-05ad75a1cacb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81512) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:46:45.319904 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988b5dff-2813-4261-b300-05ad75a1cacb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:45.321371 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988b5dff-2813-4261-b300-05ad75a1cacb None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:45.334034 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 315/624] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Aug 30 15:46:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:46:45.334871 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988b5dff-2813-4261-b300-05ad75a1cacb None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:45.382493 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 310/625] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 30 15:46:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 15:46:45.383461 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4935ff85-1d06-43bd-9327-50590d810bd5 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:45.413331 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:45.444342 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988b5dff-2813-4261-b300-05ad75a1cacb None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:45.479663 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:45.484703 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1e191a-897b-44b0-823d-f31a221f96c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:45.486168 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1e191a-897b-44b0-823d-f31a221f96c2 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:45.486888 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1e191a-897b-44b0-823d-f31a221f96c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:45.487310 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1e191a-897b-44b0-823d-f31a221f96c2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:45.490705 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4935ff85-1d06-43bd-9327-50590d810bd5 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:45.502354 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1e191a-897b-44b0-823d-f31a221f96c2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 15:46:45.510428 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application [None req-cc1e191a-897b-44b0-823d-f31a221f96c2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 15:46:45.510428 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:46:45.510428 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:46:45.510428 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:46:45.510428 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:46:45.510428 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:46:45.510428 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:46:45.510428 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:46:45.510428 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:46:45.510428 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:46:45.510428 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:46:45.510428 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:46:45.510428 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 15:46:45.510428 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 15:46:45.510428 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 15:46:45.510428 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 15:46:45.511543 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 15:46:45.511543 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 15:46:45.511543 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 15:46:45.511543 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 15:46:45.511543 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:46:45.511543 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:46:45.511543 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:46:45.511543 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:46:45.511543 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:46:45.511543 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:46:45.511543 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:46:45.511543 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application with Lock( Aug 30 15:46:45.511543 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:46:45.511543 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:46:45.511543 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:46:45.511543 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:46:45.512613 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:46:45.512613 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:46:45.512613 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:46:45.512613 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:46:45.512613 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 15:46:45.512613 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 15:46:45.512613 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 15:46:45.512613 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 15:46:45.512613 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 15:46:45.512613 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 15:46:45.512613 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 15:46:45.512613 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Aug 30 15:46:45.515565 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 316/626] 10.222.0.20 () {56 vars in 1102 bytes} [Tue Aug 30 15:46:45 2022] GET /identity/v3/roles/service => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 15:46:45.516495 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1e191a-897b-44b0-823d-f31a221f96c2 None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:45.543627 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ead628-8525-4edc-a99e-e102cd74696c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:45.544413 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ead628-8525-4edc-a99e-e102cd74696c None admin] REQUEST_METHOD: `PUT` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:45.544789 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ead628-8525-4edc-a99e-e102cd74696c None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:45.545166 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ead628-8525-4edc-a99e-e102cd74696c None admin] PATH_INFO: `/v3/projects/854fe71b49224dff9a4b1766a20ab266/users/8fdafdc39a594bb4b0aeaf2fbc84c2fb/roles/57eb8466096346b39319536c6feeb6f7` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:45.561720 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ead628-8525-4edc-a99e-e102cd74696c None admin] RBAC: Authorizing `identity:create_grant(project_id=854fe71b49224dff9a4b1766a20ab266, user_id=8fdafdc39a594bb4b0aeaf2fbc84c2fb, role_id=57eb8466096346b39319536c6feeb6f7)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:45.563676 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ead628-8525-4edc-a99e-e102cd74696c None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:45.587547 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 311/627] 10.222.0.20 () {58 vars in 1439 bytes} [Tue Aug 30 15:46:45 2022] PUT /identity/v3/projects/854fe71b49224dff9a4b1766a20ab266/users/8fdafdc39a594bb4b0aeaf2fbc84c2fb/roles/57eb8466096346b39319536c6feeb6f7 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:46:45.589107 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ead628-8525-4edc-a99e-e102cd74696c None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:45.663822 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1e191a-897b-44b0-823d-f31a221f96c2 None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:45.702116 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed32ef49-4680-4d6c-92f9-b113930fcee0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:45.703061 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed32ef49-4680-4d6c-92f9-b113930fcee0 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:45.703492 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed32ef49-4680-4d6c-92f9-b113930fcee0 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:45.703907 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed32ef49-4680-4d6c-92f9-b113930fcee0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:45.706355 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:45.717334 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed32ef49-4680-4d6c-92f9-b113930fcee0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 15:46:45.722712 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application [None req-ed32ef49-4680-4d6c-92f9-b113930fcee0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 15:46:45.722712 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:46:45.722712 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:46:45.722712 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:46:45.722712 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:46:45.722712 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:46:45.722712 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:46:45.722712 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:46:45.722712 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:46:45.722712 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:46:45.722712 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:46:45.722712 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:46:45.722712 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 15:46:45.722712 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 15:46:45.722712 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 15:46:45.722712 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 15:46:45.723920 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 15:46:45.723920 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 15:46:45.723920 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 15:46:45.723920 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 15:46:45.723920 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:46:45.723920 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:46:45.723920 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:46:45.723920 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:46:45.723920 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:46:45.723920 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:46:45.723920 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:46:45.723920 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application with Lock( Aug 30 15:46:45.723920 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:46:45.723920 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:46:45.723920 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:46:45.723920 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:46:45.724785 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:46:45.724785 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:46:45.724785 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:46:45.724785 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:46:45.724785 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 15:46:45.724785 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 15:46:45.724785 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 15:46:45.724785 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 15:46:45.724785 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 15:46:45.724785 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 15:46:45.724785 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 15:46:45.724785 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Aug 30 15:46:45.727487 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 317/628] 10.222.0.20 () {56 vars in 1102 bytes} [Tue Aug 30 15:46:45 2022] GET /identity/v3/roles/service => generated 85 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 15:46:45.728602 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed32ef49-4680-4d6c-92f9-b113930fcee0 None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:45.741543 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ead628-8525-4edc-a99e-e102cd74696c None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:45.781982 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c6efa7-910a-4f12-af72-d1e7f535ee20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:45.782776 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c6efa7-910a-4f12-af72-d1e7f535ee20 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:45.783128 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c6efa7-910a-4f12-af72-d1e7f535ee20 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:45.783467 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c6efa7-910a-4f12-af72-d1e7f535ee20 None admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:45.784268 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c6efa7-910a-4f12-af72-d1e7f535ee20 None admin] RBAC: Adding query filter params (name=service) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:46:45.784800 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c6efa7-910a-4f12-af72-d1e7f535ee20 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:45.786379 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c6efa7-910a-4f12-af72-d1e7f535ee20 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:45.796453 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 312/629] 10.222.0.20 () {56 vars in 1103 bytes} [Tue Aug 30 15:46:45 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:46:45.799882 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c6efa7-910a-4f12-af72-d1e7f535ee20 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:45.806645 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:45.843652 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed32ef49-4680-4d6c-92f9-b113930fcee0 None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:45.884035 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3720a52b-25e3-4359-8a1e-7f24d6867fce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:45.885080 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3720a52b-25e3-4359-8a1e-7f24d6867fce None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:45.885394 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3720a52b-25e3-4359-8a1e-7f24d6867fce None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:45.885665 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3720a52b-25e3-4359-8a1e-7f24d6867fce None admin] PATH_INFO: `/v3/roles/service` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:45.896415 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3720a52b-25e3-4359-8a1e-7f24d6867fce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 15:46:45.901258 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application [None req-3720a52b-25e3-4359-8a1e-7f24d6867fce None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 15:46:45.901258 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:46:45.901258 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:46:45.901258 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:46:45.901258 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:46:45.901258 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:46:45.901258 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:46:45.901258 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:46:45.901258 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:46:45.901258 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:46:45.901258 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:46:45.901258 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:46:45.901258 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 15:46:45.901258 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 15:46:45.901258 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 15:46:45.901258 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 15:46:45.902148 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 15:46:45.902148 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 15:46:45.902148 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 15:46:45.902148 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 15:46:45.902148 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:46:45.902148 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:46:45.902148 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:46:45.902148 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:46:45.902148 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:46:45.902148 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:46:45.902148 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:46:45.902148 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application with Lock( Aug 30 15:46:45.902148 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:46:45.902148 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:46:45.902148 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:46:45.902148 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:46:45.902994 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:46:45.902994 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:46:45.902994 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:46:45.902994 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:46:45.902994 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 15:46:45.902994 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 15:46:45.902994 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 15:46:45.902994 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 15:46:45.902994 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 15:46:45.902994 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 15:46:45.902994 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 15:46:45.902994 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Aug 30 15:46:45.905354 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 318/630] 10.222.0.20 () {56 vars in 1102 bytes} [Tue Aug 30 15:46:45 2022] GET /identity/v3/roles/service => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 15:46:45.906555 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3720a52b-25e3-4359-8a1e-7f24d6867fce None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:45.914876 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c6efa7-910a-4f12-af72-d1e7f535ee20 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:45.947868 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbacd6bd-a2e9-421a-92c5-938faad80a7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:45.948481 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbacd6bd-a2e9-421a-92c5-938faad80a7e None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:45.948759 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbacd6bd-a2e9-421a-92c5-938faad80a7e None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:45.949030 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbacd6bd-a2e9-421a-92c5-938faad80a7e None admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:45.949730 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbacd6bd-a2e9-421a-92c5-938faad80a7e None admin] RBAC: Adding query filter params (name=service) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:46:45.950031 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbacd6bd-a2e9-421a-92c5-938faad80a7e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:45.951415 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbacd6bd-a2e9-421a-92c5-938faad80a7e None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:45.959754 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 313/631] 10.222.0.20 () {56 vars in 1103 bytes} [Tue Aug 30 15:46:45 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:46:45.961046 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc98a708-89bc-4de8-b9b4-4a124da0979a None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:45.961363 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc98a708-89bc-4de8-b9b4-4a124da0979a None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:45.961635 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc98a708-89bc-4de8-b9b4-4a124da0979a None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:45.965468 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 314/632] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 30 15:46:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 15:46:45.965468 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc98a708-89bc-4de8-b9b4-4a124da0979a None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:45.975958 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:46.025994 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3720a52b-25e3-4359-8a1e-7f24d6867fce None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:46.037843 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:46.060975 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c58d916-0ab6-478f-a502-b558d674a3f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:46.061559 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c58d916-0ab6-478f-a502-b558d674a3f6 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:46.061838 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c58d916-0ab6-478f-a502-b558d674a3f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:46.062106 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c58d916-0ab6-478f-a502-b558d674a3f6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:46.072883 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c58d916-0ab6-478f-a502-b558d674a3f6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:46.074526 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c58d916-0ab6-478f-a502-b558d674a3f6 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:46.076136 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc98a708-89bc-4de8-b9b4-4a124da0979a None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:46.089926 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application [None req-5c58d916-0ab6-478f-a502-b558d674a3f6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 15:46:46.089926 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:46:46.089926 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:46:46.089926 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:46:46.089926 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:46:46.089926 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:46:46.089926 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:46:46.089926 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:46:46.089926 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:46:46.089926 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:46:46.089926 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:46:46.089926 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:46:46.089926 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 15:46:46.089926 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 15:46:46.089926 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 15:46:46.089926 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 15:46:46.090901 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:46:46.090901 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:46:46.090901 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:46:46.090901 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:46:46.090901 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:46:46.090901 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:46:46.090901 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:46:46.090901 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application with Lock( Aug 30 15:46:46.090901 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:46:46.090901 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:46:46.090901 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:46:46.090901 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:46:46.090901 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:46:46.090901 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:46:46.090901 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:46:46.090901 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:46:46.090901 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 15:46:46.091799 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 15:46:46.091799 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 15:46:46.091799 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Aug 30 15:46:46.091799 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 319/633] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 30 15:46:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 15:46:46.091799 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c58d916-0ab6-478f-a502-b558d674a3f6 None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:46.119017 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33849a42-b90b-423b-99f8-132a90f62c3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:46.119774 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33849a42-b90b-423b-99f8-132a90f62c3d None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:46.120067 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33849a42-b90b-423b-99f8-132a90f62c3d None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:46.120367 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33849a42-b90b-423b-99f8-132a90f62c3d None admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:46.121954 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33849a42-b90b-423b-99f8-132a90f62c3d None admin] RBAC: Adding query filter params (name=service) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:46:46.122267 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33849a42-b90b-423b-99f8-132a90f62c3d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:46.124122 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33849a42-b90b-423b-99f8-132a90f62c3d None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:46.134911 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 315/634] 10.222.0.20 () {56 vars in 1103 bytes} [Tue Aug 30 15:46:45 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:46:46.136031 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f4a38d-0a44-4d54-b818-6794eae5d857 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:46.136460 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f4a38d-0a44-4d54-b818-6794eae5d857 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:46.136732 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f4a38d-0a44-4d54-b818-6794eae5d857 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:46.154679 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-a9f4a38d-0a44-4d54-b818-6794eae5d857 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:46:46.155927 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-a9f4a38d-0a44-4d54-b818-6794eae5d857 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:46:46.199299 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c58d916-0ab6-478f-a502-b558d674a3f6 None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:46.207677 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:46.229389 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 316/635] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 30 15:46:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 15:46:46.229389 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f4a38d-0a44-4d54-b818-6794eae5d857 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:46.241856 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18344952-475d-42fc-b70c-d0586043bc8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:46.242495 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18344952-475d-42fc-b70c-d0586043bc8f None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:46.242771 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18344952-475d-42fc-b70c-d0586043bc8f None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:46.243051 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18344952-475d-42fc-b70c-d0586043bc8f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:46.256474 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18344952-475d-42fc-b70c-d0586043bc8f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:46.258130 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18344952-475d-42fc-b70c-d0586043bc8f None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:46.272907 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application [None req-18344952-475d-42fc-b70c-d0586043bc8f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 15:46:46.272907 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:46:46.272907 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:46:46.272907 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:46:46.272907 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:46:46.272907 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:46:46.272907 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:46:46.272907 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:46:46.272907 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:46:46.272907 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:46:46.272907 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:46:46.272907 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:46:46.272907 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 15:46:46.272907 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 15:46:46.272907 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 15:46:46.272907 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 15:46:46.278831 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:46:46.278831 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:46:46.278831 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:46:46.278831 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:46:46.278831 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:46:46.278831 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:46:46.278831 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:46:46.278831 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application with Lock( Aug 30 15:46:46.278831 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:46:46.278831 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:46:46.278831 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:46:46.278831 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:46:46.278831 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:46:46.278831 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:46:46.278831 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:46:46.278831 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:46:46.278831 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 15:46:46.280052 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 15:46:46.280052 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 15:46:46.280052 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Aug 30 15:46:46.280052 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 320/636] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 30 15:46:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 15:46:46.280052 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18344952-475d-42fc-b70c-d0586043bc8f None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:46.316038 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:46.371036 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f4a38d-0a44-4d54-b818-6794eae5d857 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:46.382896 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18344952-475d-42fc-b70c-d0586043bc8f None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:46.418989 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee2b6a4-76a1-4c12-92d2-54b064c723fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:46.419758 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee2b6a4-76a1-4c12-92d2-54b064c723fa None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:46.420159 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee2b6a4-76a1-4c12-92d2-54b064c723fa None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:46.420508 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee2b6a4-76a1-4c12-92d2-54b064c723fa None admin] PATH_INFO: `/v3/domains` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:46.421364 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee2b6a4-76a1-4c12-92d2-54b064c723fa None admin] RBAC: Adding query filter params (name=Default) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:46:46.421769 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee2b6a4-76a1-4c12-92d2-54b064c723fa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:46.421958 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562d766a-5f4b-42e6-a88c-45e6be4c6611 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:46.422727 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562d766a-5f4b-42e6-a88c-45e6be4c6611 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:46.423158 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562d766a-5f4b-42e6-a88c-45e6be4c6611 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:46.423565 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee2b6a4-76a1-4c12-92d2-54b064c723fa None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:46.423677 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562d766a-5f4b-42e6-a88c-45e6be4c6611 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:46.436995 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 317/637] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Aug 30 15:46:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:46:46.437967 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562d766a-5f4b-42e6-a88c-45e6be4c6611 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:46.439715 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562d766a-5f4b-42e6-a88c-45e6be4c6611 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:46.440667 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee2b6a4-76a1-4c12-92d2-54b064c723fa None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:46.456668 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application [None req-562d766a-5f4b-42e6-a88c-45e6be4c6611 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 15:46:46.456668 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:46:46.456668 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:46:46.456668 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:46:46.456668 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:46:46.456668 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:46:46.456668 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:46:46.456668 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:46:46.456668 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:46:46.456668 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:46:46.456668 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:46:46.456668 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:46:46.456668 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 15:46:46.456668 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 15:46:46.456668 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 15:46:46.456668 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 15:46:46.457711 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:46:46.457711 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:46:46.457711 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:46:46.457711 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:46:46.457711 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:46:46.457711 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:46:46.457711 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:46:46.457711 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application with Lock( Aug 30 15:46:46.457711 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:46:46.457711 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:46:46.457711 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:46:46.457711 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:46:46.457711 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:46:46.457711 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:46:46.457711 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:46:46.457711 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:46:46.457711 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 15:46:46.458742 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 15:46:46.458742 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 15:46:46.458742 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Aug 30 15:46:46.461092 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 321/638] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 30 15:46:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 15:46:46.462132 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562d766a-5f4b-42e6-a88c-45e6be4c6611 None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:46.476401 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:46.567699 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee2b6a4-76a1-4c12-92d2-54b064c723fa None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:46.569025 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562d766a-5f4b-42e6-a88c-45e6be4c6611 None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:46.587822 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:46.603383 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62340f45-364d-4e24-85de-bf6eaab9267b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:46.604016 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62340f45-364d-4e24-85de-bf6eaab9267b None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:46.604290 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62340f45-364d-4e24-85de-bf6eaab9267b None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:46.604553 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62340f45-364d-4e24-85de-bf6eaab9267b None admin] PATH_INFO: `/v3/users/nova` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:46.608369 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de95011e-38a3-4031-ad49-13f95ba002a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:46.609078 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de95011e-38a3-4031-ad49-13f95ba002a4 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:46.610871 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de95011e-38a3-4031-ad49-13f95ba002a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:46.611255 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de95011e-38a3-4031-ad49-13f95ba002a4 None admin] PATH_INFO: `/v3/domains` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:46.612026 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de95011e-38a3-4031-ad49-13f95ba002a4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:46:46.612129 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62340f45-364d-4e24-85de-bf6eaab9267b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:46.612491 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de95011e-38a3-4031-ad49-13f95ba002a4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:46.613604 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62340f45-364d-4e24-85de-bf6eaab9267b None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:46.614069 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de95011e-38a3-4031-ad49-13f95ba002a4 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:46.624985 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application [None req-62340f45-364d-4e24-85de-bf6eaab9267b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 30 15:46:46.624985 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:46:46.624985 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:46:46.624985 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:46:46.624985 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:46:46.624985 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:46:46.624985 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:46:46.624985 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:46:46.624985 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:46:46.624985 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:46:46.624985 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:46:46.624985 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:46:46.624985 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 15:46:46.624985 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 15:46:46.624985 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 15:46:46.624985 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 15:46:46.626000 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:46:46.626000 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:46:46.626000 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 15:46:46.626000 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 15:46:46.626000 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 15:46:46.626000 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 15:46:46.626000 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:46:46.626000 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:46:46.626000 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:46:46.626000 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:46:46.626000 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:46:46.626000 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application with Lock( Aug 30 15:46:46.626000 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:46:46.626000 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:46:46.626000 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:46:46.626000 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:46:46.626000 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:46:46.626000 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:46:46.626972 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:46:46.626972 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:46:46.626972 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 15:46:46.626972 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 15:46:46.626972 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 15:46:46.626972 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 15:46:46.626972 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 15:46:46.626972 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 15:46:46.626972 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 30 15:46:46.626972 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Aug 30 15:46:46.631014 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 322/639] 10.222.0.20 () {56 vars in 1093 bytes} [Tue Aug 30 15:46:46 2022] GET /identity/v3/users/nova => generated 82 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 15:46:46.635497 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 318/640] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Aug 30 15:46:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:46:46.635623 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62340f45-364d-4e24-85de-bf6eaab9267b None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:46.635726 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de95011e-38a3-4031-ad49-13f95ba002a4 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:46.708804 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:46.748384 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62340f45-364d-4e24-85de-bf6eaab9267b None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:46.749260 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de95011e-38a3-4031-ad49-13f95ba002a4 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:46.785166 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d96cec-c894-4d7a-89fb-f68f183a9bd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:46.785803 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d96cec-c894-4d7a-89fb-f68f183a9bd7 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:46.786083 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d96cec-c894-4d7a-89fb-f68f183a9bd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:46.786352 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d96cec-c894-4d7a-89fb-f68f183a9bd7 None admin] PATH_INFO: `/v3/domains` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:46.786991 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d96cec-c894-4d7a-89fb-f68f183a9bd7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81512) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:46:46.787293 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d96cec-c894-4d7a-89fb-f68f183a9bd7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:46.787739 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8882d107-64c9-4fce-83d8-c80b4608a6d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:46.788377 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8882d107-64c9-4fce-83d8-c80b4608a6d6 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:46.788651 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8882d107-64c9-4fce-83d8-c80b4608a6d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:46.788745 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d96cec-c894-4d7a-89fb-f68f183a9bd7 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:46.788909 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8882d107-64c9-4fce-83d8-c80b4608a6d6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:46.799931 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8882d107-64c9-4fce-83d8-c80b4608a6d6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 15:46:46.804953 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 323/641] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Aug 30 15:46:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:46:46.804953 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d96cec-c894-4d7a-89fb-f68f183a9bd7 None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:46.805189 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-8882d107-64c9-4fce-83d8-c80b4608a6d6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 15:46:46.805189 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:46:46.805189 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:46:46.805189 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:46:46.805189 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:46:46.805189 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:46:46.805189 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:46:46.805189 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:46:46.805189 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:46:46.805189 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:46:46.805189 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:46:46.805189 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:46:46.805189 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 15:46:46.805189 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 15:46:46.805189 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 15:46:46.805189 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 15:46:46.806399 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 15:46:46.806399 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 15:46:46.806399 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 15:46:46.806399 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 15:46:46.806399 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:46:46.806399 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:46:46.806399 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:46:46.806399 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:46:46.806399 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:46:46.806399 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:46:46.806399 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:46:46.806399 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Aug 30 15:46:46.806399 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:46:46.806399 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:46:46.806399 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:46:46.806399 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:46:46.807257 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:46:46.807257 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:46:46.807257 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:46:46.807257 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:46:46.807257 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 15:46:46.807257 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 15:46:46.807257 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 15:46:46.807257 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 15:46:46.807257 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 15:46:46.807257 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 15:46:46.807257 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 15:46:46.807257 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Aug 30 15:46:46.807257 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 319/642] 10.222.0.20 () {56 vars in 1102 bytes} [Tue Aug 30 15:46:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 15:46:46.807996 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8882d107-64c9-4fce-83d8-c80b4608a6d6 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:46.850482 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:46.911481 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d96cec-c894-4d7a-89fb-f68f183a9bd7 None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:46.912726 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8882d107-64c9-4fce-83d8-c80b4608a6d6 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:46.940820 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:46.948222 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dff4793-5e4c-4dca-aaf3-1cca49f68c14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:46.948837 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dff4793-5e4c-4dca-aaf3-1cca49f68c14 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:46.949114 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dff4793-5e4c-4dca-aaf3-1cca49f68c14 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:46.949429 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dff4793-5e4c-4dca-aaf3-1cca49f68c14 None admin] PATH_INFO: `/v3/users` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:46.950159 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dff4793-5e4c-4dca-aaf3-1cca49f68c14 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81512) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:46:46.950466 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dff4793-5e4c-4dca-aaf3-1cca49f68c14 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:46.950560 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5105dffe-0414-4a4c-80da-7bdd10863f9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:46.951161 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5105dffe-0414-4a4c-80da-7bdd10863f9d None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:46.951440 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5105dffe-0414-4a4c-80da-7bdd10863f9d None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:46.952326 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5105dffe-0414-4a4c-80da-7bdd10863f9d None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:46.952417 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dff4793-5e4c-4dca-aaf3-1cca49f68c14 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:46.960309 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5105dffe-0414-4a4c-80da-7bdd10863f9d None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:46.961783 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5105dffe-0414-4a4c-80da-7bdd10863f9d None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:46.976192 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 324/643] 10.222.0.20 () {56 vars in 1133 bytes} [Tue Aug 30 15:46:46 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:46:46.976192 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dff4793-5e4c-4dca-aaf3-1cca49f68c14 None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:46.976461 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-5105dffe-0414-4a4c-80da-7bdd10863f9d None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 30 15:46:46.976461 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:46:46.976461 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:46:46.976461 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:46:46.976461 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:46:46.976461 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:46:46.976461 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:46:46.976461 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:46:46.976461 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:46:46.976461 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:46:46.976461 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:46:46.976461 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:46:46.976461 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 15:46:46.976461 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 15:46:46.976461 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 15:46:46.976461 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 15:46:46.977909 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:46:46.977909 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:46:46.977909 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 15:46:46.977909 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 15:46:46.977909 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 15:46:46.977909 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 15:46:46.977909 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:46:46.977909 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:46:46.977909 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:46:46.977909 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:46:46.977909 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:46:46.977909 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Aug 30 15:46:46.977909 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:46:46.977909 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:46:46.977909 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:46:46.977909 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:46:46.977909 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:46:46.977909 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:46:46.979002 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:46:46.979002 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:46:46.979002 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 15:46:46.979002 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 15:46:46.979002 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 15:46:46.979002 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 15:46:46.979002 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 15:46:46.979002 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 15:46:46.979002 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 30 15:46:46.979002 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Aug 30 15:46:46.979002 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 320/644] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Aug 30 15:46:46 2022] GET /identity/v3/users/cinder => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 15:46:46.979002 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5105dffe-0414-4a4c-80da-7bdd10863f9d None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:47.075303 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:47.079263 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5105dffe-0414-4a4c-80da-7bdd10863f9d None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:47.084955 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dff4793-5e4c-4dca-aaf3-1cca49f68c14 None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:47.114167 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31234c7c-829b-466a-b753-9dcbf5e2b01b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:47.114705 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31234c7c-829b-466a-b753-9dcbf5e2b01b None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:47.114983 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31234c7c-829b-466a-b753-9dcbf5e2b01b None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:47.115251 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31234c7c-829b-466a-b753-9dcbf5e2b01b None admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:47.115918 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31234c7c-829b-466a-b753-9dcbf5e2b01b None admin] RBAC: Adding query filter params (name=service) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:46:47.116217 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31234c7c-829b-466a-b753-9dcbf5e2b01b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:47.117628 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31234c7c-829b-466a-b753-9dcbf5e2b01b None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:47.122579 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03e8e08-8e54-4f56-b9e7-7d4ab941a760 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:47.123378 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03e8e08-8e54-4f56-b9e7-7d4ab941a760 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:47.123856 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03e8e08-8e54-4f56-b9e7-7d4ab941a760 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:47.124273 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03e8e08-8e54-4f56-b9e7-7d4ab941a760 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:47.126309 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 321/645] 10.222.0.20 () {56 vars in 1103 bytes} [Tue Aug 30 15:46:46 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:46:47.127020 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31234c7c-829b-466a-b753-9dcbf5e2b01b None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:47.137300 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03e8e08-8e54-4f56-b9e7-7d4ab941a760 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:47.138990 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03e8e08-8e54-4f56-b9e7-7d4ab941a760 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:47.151626 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application [None req-c03e8e08-8e54-4f56-b9e7-7d4ab941a760 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 30 15:46:47.151626 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:46:47.151626 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:46:47.151626 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:46:47.151626 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:46:47.151626 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:46:47.151626 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:46:47.151626 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:46:47.151626 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:46:47.151626 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:46:47.151626 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:46:47.151626 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:46:47.151626 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 15:46:47.151626 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 15:46:47.151626 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 15:46:47.151626 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 15:46:47.157789 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:46:47.157789 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:46:47.157789 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 15:46:47.157789 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 15:46:47.157789 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 15:46:47.157789 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 15:46:47.157789 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:46:47.157789 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:46:47.157789 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:46:47.157789 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:46:47.157789 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:46:47.157789 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application with Lock( Aug 30 15:46:47.157789 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:46:47.157789 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:46:47.157789 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:46:47.157789 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:46:47.157789 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:46:47.157789 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:46:47.158981 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:46:47.158981 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:46:47.158981 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 15:46:47.158981 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 15:46:47.158981 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 15:46:47.158981 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 15:46:47.158981 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 15:46:47.158981 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 15:46:47.158981 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 30 15:46:47.158981 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Aug 30 15:46:47.158981 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 325/646] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Aug 30 15:46:46 2022] GET /identity/v3/users/glance => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 15:46:47.158981 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03e8e08-8e54-4f56-b9e7-7d4ab941a760 None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:47.208677 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:47.247421 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31234c7c-829b-466a-b753-9dcbf5e2b01b None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:47.260824 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03e8e08-8e54-4f56-b9e7-7d4ab941a760 None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:47.282923 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee95a03c-4256-448b-aeea-d5a0296da31a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:47.283625 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee95a03c-4256-448b-aeea-d5a0296da31a None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:47.283906 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee95a03c-4256-448b-aeea-d5a0296da31a None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:47.284174 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee95a03c-4256-448b-aeea-d5a0296da31a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:47.294394 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0d0654-c9b1-478b-9fdf-653cfc7bed3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:47.295001 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0d0654-c9b1-478b-9fdf-653cfc7bed3a None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:47.295289 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0d0654-c9b1-478b-9fdf-653cfc7bed3a None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:47.295581 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0d0654-c9b1-478b-9fdf-653cfc7bed3a None admin] PATH_INFO: `/v3/users` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:47.296321 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0d0654-c9b1-478b-9fdf-653cfc7bed3a None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81512) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:46:47.296628 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0d0654-c9b1-478b-9fdf-653cfc7bed3a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:47.298114 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0d0654-c9b1-478b-9fdf-653cfc7bed3a None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:47.314349 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:47.315060 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 326/647] 10.222.0.20 () {56 vars in 1137 bytes} [Tue Aug 30 15:46:47 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:46:47.315959 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0d0654-c9b1-478b-9fdf-653cfc7bed3a None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:47.366569 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee95a03c-4256-448b-aeea-d5a0296da31a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:47.368655 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee95a03c-4256-448b-aeea-d5a0296da31a None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:47.426760 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0d0654-c9b1-478b-9fdf-653cfc7bed3a None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:47.435108 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:47.465456 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8129e17-d7ae-4a55-bd9a-2f9a6733cd67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:47.466085 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8129e17-d7ae-4a55-bd9a-2f9a6733cd67 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:47.466371 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8129e17-d7ae-4a55-bd9a-2f9a6733cd67 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:47.466616 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8129e17-d7ae-4a55-bd9a-2f9a6733cd67 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:47.478765 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 322/648] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Aug 30 15:46:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 15:46:47.478765 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee95a03c-4256-448b-aeea-d5a0296da31a None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:47.482256 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8129e17-d7ae-4a55-bd9a-2f9a6733cd67 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:47.483748 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8129e17-d7ae-4a55-bd9a-2f9a6733cd67 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:47.499408 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application [None req-c8129e17-d7ae-4a55-bd9a-2f9a6733cd67 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 15:46:47.499408 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:46:47.499408 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:46:47.499408 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:46:47.499408 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:46:47.499408 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:46:47.499408 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:46:47.499408 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:46:47.499408 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:46:47.499408 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:46:47.499408 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:46:47.499408 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:46:47.499408 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 15:46:47.499408 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 15:46:47.499408 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 15:46:47.499408 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 15:46:47.500552 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:46:47.500552 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:46:47.500552 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:46:47.500552 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:46:47.500552 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:46:47.500552 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:46:47.500552 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:46:47.500552 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application with Lock( Aug 30 15:46:47.500552 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:46:47.500552 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:46:47.500552 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:46:47.500552 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:46:47.500552 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:46:47.500552 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:46:47.500552 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:46:47.500552 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:46:47.500552 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 15:46:47.501805 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 15:46:47.501805 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 15:46:47.501805 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Aug 30 15:46:47.501805 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 327/649] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 30 15:46:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 15:46:47.501805 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8129e17-d7ae-4a55-bd9a-2f9a6733cd67 None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:47.542907 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:47.586472 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee95a03c-4256-448b-aeea-d5a0296da31a None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:47.605548 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8129e17-d7ae-4a55-bd9a-2f9a6733cd67 None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:47.624019 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e48b86b-929b-4903-8d9a-2129c04c2c7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:47.624559 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e48b86b-929b-4903-8d9a-2129c04c2c7a None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:47.624845 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e48b86b-929b-4903-8d9a-2129c04c2c7a None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:47.625114 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e48b86b-929b-4903-8d9a-2129c04c2c7a None admin] PATH_INFO: `/v3/users` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:47.628750 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e48b86b-929b-4903-8d9a-2129c04c2c7a None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:46:47.629109 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e48b86b-929b-4903-8d9a-2129c04c2c7a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:47.630661 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e48b86b-929b-4903-8d9a-2129c04c2c7a None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:47.639626 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ebbd42-3229-419b-bdd6-24437c3c9eed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:47.640233 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ebbd42-3229-419b-bdd6-24437c3c9eed None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:47.640512 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ebbd42-3229-419b-bdd6-24437c3c9eed None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:47.640784 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ebbd42-3229-419b-bdd6-24437c3c9eed None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:47.648967 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 323/650] 10.222.0.20 () {56 vars in 1137 bytes} [Tue Aug 30 15:46:47 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:46:47.650315 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e48b86b-929b-4903-8d9a-2129c04c2c7a None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:47.670782 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:47.712897 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ebbd42-3229-419b-bdd6-24437c3c9eed None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:47.714389 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ebbd42-3229-419b-bdd6-24437c3c9eed None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:47.750211 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e48b86b-929b-4903-8d9a-2129c04c2c7a None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:47.767234 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:47.783705 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4104b855-fcd4-4a28-b829-b4749c18d385 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:47.784331 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4104b855-fcd4-4a28-b829-b4749c18d385 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:47.784604 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4104b855-fcd4-4a28-b829-b4749c18d385 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:47.784868 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4104b855-fcd4-4a28-b829-b4749c18d385 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:47.795160 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4104b855-fcd4-4a28-b829-b4749c18d385 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:47.796653 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4104b855-fcd4-4a28-b829-b4749c18d385 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:47.810833 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-4104b855-fcd4-4a28-b829-b4749c18d385 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 15:46:47.810833 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:46:47.810833 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:46:47.810833 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:46:47.810833 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:46:47.810833 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:46:47.810833 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:46:47.810833 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:46:47.810833 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:46:47.810833 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:46:47.810833 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:46:47.810833 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:46:47.810833 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 15:46:47.810833 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 15:46:47.810833 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 15:46:47.810833 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 15:46:47.811958 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:46:47.811958 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:46:47.811958 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:46:47.811958 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:46:47.811958 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:46:47.811958 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:46:47.811958 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:46:47.811958 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Aug 30 15:46:47.811958 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:46:47.811958 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:46:47.811958 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:46:47.811958 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:46:47.811958 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:46:47.811958 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:46:47.811958 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:46:47.811958 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:46:47.811958 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 15:46:47.813243 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 15:46:47.813243 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 15:46:47.813243 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Aug 30 15:46:47.813423 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 328/651] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Aug 30 15:46:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 15:46:47.813496 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 324/652] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 30 15:46:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 15:46:47.813496 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940db93d-0364-464e-8b2e-5b1a5fd99d11 None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:47.813688 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940db93d-0364-464e-8b2e-5b1a5fd99d11 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:47.813971 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940db93d-0364-464e-8b2e-5b1a5fd99d11 None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:47.814654 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ebbd42-3229-419b-bdd6-24437c3c9eed None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:47.817264 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 325/653] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 30 15:46:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 15:46:47.817264 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940db93d-0364-464e-8b2e-5b1a5fd99d11 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:47.874836 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:47.915479 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ebbd42-3229-419b-bdd6-24437c3c9eed None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:47.919209 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940db93d-0364-464e-8b2e-5b1a5fd99d11 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:47.949195 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7152c1c-3d6c-4d3c-b2b5-2332514c9ce4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:47.949839 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7152c1c-3d6c-4d3c-b2b5-2332514c9ce4 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:47.950102 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7152c1c-3d6c-4d3c-b2b5-2332514c9ce4 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:47.950366 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7152c1c-3d6c-4d3c-b2b5-2332514c9ce4 None admin] PATH_INFO: `/v3/domains` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:47.950996 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7152c1c-3d6c-4d3c-b2b5-2332514c9ce4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81512) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:46:47.951291 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7152c1c-3d6c-4d3c-b2b5-2332514c9ce4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:47.952033 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac0fa28-ae2f-46ab-85e4-9a53792723d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:47.952642 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac0fa28-ae2f-46ab-85e4-9a53792723d1 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:47.952744 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7152c1c-3d6c-4d3c-b2b5-2332514c9ce4 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:47.952922 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac0fa28-ae2f-46ab-85e4-9a53792723d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:47.953190 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac0fa28-ae2f-46ab-85e4-9a53792723d1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:47.964189 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 329/654] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Aug 30 15:46:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:46:47.964802 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7152c1c-3d6c-4d3c-b2b5-2332514c9ce4 None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:47.997833 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:48.025830 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac0fa28-ae2f-46ab-85e4-9a53792723d1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:48.027292 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac0fa28-ae2f-46ab-85e4-9a53792723d1 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:48.066388 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7152c1c-3d6c-4d3c-b2b5-2332514c9ce4 None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:48.083285 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:48.100111 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9b2dee-a70a-4beb-80aa-844815bdf0b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:48.100740 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9b2dee-a70a-4beb-80aa-844815bdf0b1 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:48.101041 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9b2dee-a70a-4beb-80aa-844815bdf0b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:48.101324 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9b2dee-a70a-4beb-80aa-844815bdf0b1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:48.112618 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9b2dee-a70a-4beb-80aa-844815bdf0b1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:48.114112 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9b2dee-a70a-4beb-80aa-844815bdf0b1 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:48.128605 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application [None req-2a9b2dee-a70a-4beb-80aa-844815bdf0b1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 15:46:48.128605 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:46:48.128605 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:46:48.128605 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:46:48.128605 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:46:48.128605 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:46:48.128605 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:46:48.128605 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:46:48.128605 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:46:48.128605 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:46:48.128605 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:46:48.128605 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:46:48.128605 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 15:46:48.128605 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 15:46:48.128605 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 15:46:48.128605 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 15:46:48.129736 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:46:48.129736 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:46:48.129736 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:46:48.129736 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:46:48.129736 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:46:48.129736 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:46:48.129736 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:46:48.129736 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application with Lock( Aug 30 15:46:48.129736 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:46:48.129736 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:46:48.129736 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:46:48.129736 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:46:48.129736 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:46:48.129736 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:46:48.129736 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:46:48.129736 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:46:48.129736 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 15:46:48.130656 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 15:46:48.130656 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 15:46:48.130656 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Aug 30 15:46:48.130656 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 330/655] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 30 15:46:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 15:46:48.130883 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9b2dee-a70a-4beb-80aa-844815bdf0b1 None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:48.133528 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 326/656] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Aug 30 15:46:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 15:46:48.135238 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944a6ce5-3535-4c5d-b66e-f1928b161427 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:48.135519 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944a6ce5-3535-4c5d-b66e-f1928b161427 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:48.135784 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944a6ce5-3535-4c5d-b66e-f1928b161427 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:48.153346 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-944a6ce5-3535-4c5d-b66e-f1928b161427 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:46:48.154529 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-944a6ce5-3535-4c5d-b66e-f1928b161427 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:46:48.213986 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:48.219561 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 327/657] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 30 15:46:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 15:46:48.220357 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944a6ce5-3535-4c5d-b66e-f1928b161427 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:48.232172 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9b2dee-a70a-4beb-80aa-844815bdf0b1 None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:48.268272 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344bdba8-2c2b-415e-9a5a-c0de91da20af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:48.268859 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344bdba8-2c2b-415e-9a5a-c0de91da20af None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:48.269142 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344bdba8-2c2b-415e-9a5a-c0de91da20af None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:48.269433 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344bdba8-2c2b-415e-9a5a-c0de91da20af None admin] PATH_INFO: `/v3/domains` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:48.270172 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344bdba8-2c2b-415e-9a5a-c0de91da20af None admin] RBAC: Adding query filter params (name=Default) {{(pid=81512) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:46:48.270481 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344bdba8-2c2b-415e-9a5a-c0de91da20af None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:48.271852 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344bdba8-2c2b-415e-9a5a-c0de91da20af None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:48.287980 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 331/658] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Aug 30 15:46:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:46:48.287980 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344bdba8-2c2b-415e-9a5a-c0de91da20af None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:48.315820 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:48.334840 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944a6ce5-3535-4c5d-b66e-f1928b161427 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:48.374812 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6916f6b5-02ef-450c-8632-2c03d13f1969 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:48.375435 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6916f6b5-02ef-450c-8632-2c03d13f1969 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:48.375706 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6916f6b5-02ef-450c-8632-2c03d13f1969 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:48.375971 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6916f6b5-02ef-450c-8632-2c03d13f1969 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:48.383056 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6916f6b5-02ef-450c-8632-2c03d13f1969 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:48.384506 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6916f6b5-02ef-450c-8632-2c03d13f1969 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:48.396041 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-6916f6b5-02ef-450c-8632-2c03d13f1969 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 30 15:46:48.396041 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:46:48.396041 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:46:48.396041 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:46:48.396041 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:46:48.396041 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:46:48.396041 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:46:48.396041 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:46:48.396041 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:46:48.396041 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:46:48.396041 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:46:48.396041 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:46:48.396041 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 15:46:48.396041 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 15:46:48.396041 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 15:46:48.396041 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 15:46:48.397064 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:46:48.397064 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:46:48.397064 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 15:46:48.397064 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 15:46:48.397064 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 15:46:48.397064 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 15:46:48.397064 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:46:48.397064 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:46:48.397064 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:46:48.397064 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:46:48.397064 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:46:48.397064 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Aug 30 15:46:48.397064 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:46:48.397064 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:46:48.397064 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:46:48.397064 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:46:48.397064 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:46:48.397064 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:46:48.398155 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:46:48.398155 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:46:48.398155 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 15:46:48.398155 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 15:46:48.398155 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 15:46:48.398155 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 15:46:48.398155 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 15:46:48.398155 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 15:46:48.398155 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 30 15:46:48.398155 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Aug 30 15:46:48.398155 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 328/659] 10.222.0.20 () {56 vars in 1102 bytes} [Tue Aug 30 15:46:48 2022] GET /identity/v3/users/neutron => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 15:46:48.398155 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6916f6b5-02ef-450c-8632-2c03d13f1969 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:48.406880 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344bdba8-2c2b-415e-9a5a-c0de91da20af None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:48.442037 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78badad-412a-4343-8441-624cc9a0bb9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:48.442895 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78badad-412a-4343-8441-624cc9a0bb9f None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:48.443326 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78badad-412a-4343-8441-624cc9a0bb9f None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:48.443794 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78badad-412a-4343-8441-624cc9a0bb9f None admin] PATH_INFO: `/v3/domains` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:48.444578 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78badad-412a-4343-8441-624cc9a0bb9f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81512) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:46:48.445030 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78badad-412a-4343-8441-624cc9a0bb9f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:48.446716 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78badad-412a-4343-8441-624cc9a0bb9f None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:48.458676 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 332/660] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Aug 30 15:46:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:46:48.459832 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78badad-412a-4343-8441-624cc9a0bb9f None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:48.466558 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:48.511006 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6916f6b5-02ef-450c-8632-2c03d13f1969 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:48.543453 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c422cf-8bd4-452d-8121-c01e46d5cc5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:48.544061 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c422cf-8bd4-452d-8121-c01e46d5cc5b None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:48.544334 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c422cf-8bd4-452d-8121-c01e46d5cc5b None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:48.544599 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c422cf-8bd4-452d-8121-c01e46d5cc5b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:48.553211 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:48.555305 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c422cf-8bd4-452d-8121-c01e46d5cc5b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:48.556749 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c422cf-8bd4-452d-8121-c01e46d5cc5b None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:48.563154 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78badad-412a-4343-8441-624cc9a0bb9f None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:48.570618 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-f3c422cf-8bd4-452d-8121-c01e46d5cc5b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 15:46:48.570618 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:46:48.570618 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:46:48.570618 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:46:48.570618 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:46:48.570618 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:46:48.570618 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:46:48.570618 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:46:48.570618 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:46:48.570618 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:46:48.570618 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:46:48.570618 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:46:48.570618 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 15:46:48.570618 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 15:46:48.570618 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 15:46:48.570618 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 15:46:48.571605 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:46:48.571605 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:46:48.571605 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:46:48.571605 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:46:48.571605 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:46:48.571605 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:46:48.571605 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:46:48.571605 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Aug 30 15:46:48.571605 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:46:48.571605 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:46:48.571605 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:46:48.571605 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:46:48.571605 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:46:48.571605 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:46:48.571605 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:46:48.571605 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:46:48.571605 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 15:46:48.572520 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 15:46:48.572520 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 15:46:48.572520 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Aug 30 15:46:48.573359 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 329/661] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 30 15:46:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 15:46:48.574340 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c422cf-8bd4-452d-8121-c01e46d5cc5b None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:48.597077 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c45d6a-aa3b-4938-8a25-427b6d4a44da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:48.597749 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c45d6a-aa3b-4938-8a25-427b6d4a44da None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:48.598022 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c45d6a-aa3b-4938-8a25-427b6d4a44da None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:48.598301 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c45d6a-aa3b-4938-8a25-427b6d4a44da None admin] PATH_INFO: `/v3/projects/service` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:48.605352 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c45d6a-aa3b-4938-8a25-427b6d4a44da None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:48.606809 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c45d6a-aa3b-4938-8a25-427b6d4a44da None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:48.617937 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application [None req-c5c45d6a-aa3b-4938-8a25-427b6d4a44da None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 15:46:48.617937 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:46:48.617937 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:46:48.617937 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:46:48.617937 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:46:48.617937 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:46:48.617937 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:46:48.617937 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:46:48.617937 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:46:48.617937 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:46:48.617937 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:46:48.617937 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:46:48.617937 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 15:46:48.617937 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 15:46:48.617937 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 15:46:48.619076 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 15:46:48.619076 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:46:48.619076 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:46:48.619076 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:46:48.619076 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:46:48.619076 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:46:48.619076 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:46:48.619076 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:46:48.619076 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application with Lock( Aug 30 15:46:48.619076 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:46:48.619076 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:46:48.619076 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:46:48.619076 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:46:48.619076 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:46:48.619076 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:46:48.619076 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:46:48.619076 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:46:48.620126 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 15:46:48.620126 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 15:46:48.620126 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 15:46:48.620126 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 15:46:48.620126 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 15:46:48.620126 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 15:46:48.620126 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 15:46:48.620126 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Aug 30 15:46:48.620126 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 333/662] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Aug 30 15:46:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 15:46:48.621034 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c45d6a-aa3b-4938-8a25-427b6d4a44da None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:48.678704 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c422cf-8bd4-452d-8121-c01e46d5cc5b None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:48.691699 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:48.712203 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5105d16-6612-44ae-8365-c1d486be80a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:48.712906 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5105d16-6612-44ae-8365-c1d486be80a4 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:48.713186 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5105d16-6612-44ae-8365-c1d486be80a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:48.713493 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5105d16-6612-44ae-8365-c1d486be80a4 None admin] PATH_INFO: `/v3/users` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:48.714228 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5105d16-6612-44ae-8365-c1d486be80a4 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:46:48.714533 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5105d16-6612-44ae-8365-c1d486be80a4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:48.715981 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5105d16-6612-44ae-8365-c1d486be80a4 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:48.718126 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c45d6a-aa3b-4938-8a25-427b6d4a44da None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:48.733614 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 330/663] 10.222.0.20 () {56 vars in 1139 bytes} [Tue Aug 30 15:46:48 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:46:48.734502 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5105d16-6612-44ae-8365-c1d486be80a4 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:48.757736 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0295ef-e9ee-4019-a854-fb3b27eaa27c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:48.758375 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0295ef-e9ee-4019-a854-fb3b27eaa27c None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:48.758652 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0295ef-e9ee-4019-a854-fb3b27eaa27c None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:48.758918 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0295ef-e9ee-4019-a854-fb3b27eaa27c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:48.765562 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0295ef-e9ee-4019-a854-fb3b27eaa27c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:48.767014 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0295ef-e9ee-4019-a854-fb3b27eaa27c None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:48.778298 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application [None req-ae0295ef-e9ee-4019-a854-fb3b27eaa27c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 15:46:48.778298 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:46:48.778298 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:46:48.778298 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:46:48.778298 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:46:48.778298 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:46:48.778298 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:46:48.778298 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:46:48.778298 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:46:48.778298 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:46:48.778298 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:46:48.778298 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:46:48.778298 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 15:46:48.778298 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 15:46:48.778298 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 15:46:48.779388 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 15:46:48.779388 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:46:48.779388 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:46:48.779388 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:46:48.779388 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:46:48.779388 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:46:48.779388 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:46:48.779388 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:46:48.779388 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application with Lock( Aug 30 15:46:48.779388 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:46:48.779388 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:46:48.779388 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:46:48.779388 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:46:48.779388 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:46:48.779388 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:46:48.779388 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:46:48.779388 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:46:48.780463 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 15:46:48.780463 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 15:46:48.780463 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 15:46:48.780463 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 15:46:48.780463 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 15:46:48.780463 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 15:46:48.780463 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 15:46:48.780463 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Aug 30 15:46:48.780463 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 334/664] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Aug 30 15:46:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 15:46:48.781042 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0295ef-e9ee-4019-a854-fb3b27eaa27c None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:48.792929 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:48.834773 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5105d16-6612-44ae-8365-c1d486be80a4 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:48.868306 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a178a35-28bf-46d3-b4fc-6a0a49d17572 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:48.868938 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a178a35-28bf-46d3-b4fc-6a0a49d17572 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:48.869244 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a178a35-28bf-46d3-b4fc-6a0a49d17572 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:48.869511 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a178a35-28bf-46d3-b4fc-6a0a49d17572 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:48.880467 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a178a35-28bf-46d3-b4fc-6a0a49d17572 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 15:46:48.881803 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0295ef-e9ee-4019-a854-fb3b27eaa27c None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:48.885232 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-8a178a35-28bf-46d3-b4fc-6a0a49d17572 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 15:46:48.885232 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:46:48.885232 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:46:48.885232 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:46:48.885232 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:46:48.885232 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:46:48.885232 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:46:48.885232 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:46:48.885232 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:46:48.885232 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:46:48.885232 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:46:48.885232 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:46:48.885232 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 15:46:48.885232 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 15:46:48.885232 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 15:46:48.885232 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 15:46:48.886327 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 15:46:48.886327 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 15:46:48.886327 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 15:46:48.886327 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 15:46:48.886327 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:46:48.886327 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:46:48.886327 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:46:48.886327 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:46:48.886327 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:46:48.886327 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:46:48.886327 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:46:48.886327 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Aug 30 15:46:48.886327 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:46:48.886327 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:46:48.886327 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:46:48.886327 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:46:48.887203 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:46:48.887203 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:46:48.887203 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:46:48.887203 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:46:48.887203 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 15:46:48.887203 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 15:46:48.887203 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 15:46:48.887203 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 15:46:48.887203 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 15:46:48.887203 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 15:46:48.887203 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 15:46:48.887203 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Aug 30 15:46:48.887203 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 331/665] 10.222.0.20 () {56 vars in 1102 bytes} [Tue Aug 30 15:46:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 15:46:48.887938 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a178a35-28bf-46d3-b4fc-6a0a49d17572 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:48.916618 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f929e573-ab0d-4360-9d96-ba612452a666 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:48.917151 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f929e573-ab0d-4360-9d96-ba612452a666 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:48.917351 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:48.917486 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f929e573-ab0d-4360-9d96-ba612452a666 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:48.917716 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f929e573-ab0d-4360-9d96-ba612452a666 None admin] PATH_INFO: `/v3/domains` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:48.918360 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f929e573-ab0d-4360-9d96-ba612452a666 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81512) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:46:48.918708 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f929e573-ab0d-4360-9d96-ba612452a666 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:48.920079 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f929e573-ab0d-4360-9d96-ba612452a666 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:48.932448 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 335/666] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Aug 30 15:46:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:46:48.933264 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f929e573-ab0d-4360-9d96-ba612452a666 None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:48.986530 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a178a35-28bf-46d3-b4fc-6a0a49d17572 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:49.002105 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:49.019289 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29508ae-19e9-48e1-90fb-8f0fefc9a75d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:49.019901 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29508ae-19e9-48e1-90fb-8f0fefc9a75d None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:49.020172 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29508ae-19e9-48e1-90fb-8f0fefc9a75d None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:49.020487 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29508ae-19e9-48e1-90fb-8f0fefc9a75d None admin] PATH_INFO: `/v3/projects` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:49.021186 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29508ae-19e9-48e1-90fb-8f0fefc9a75d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:46:49.021529 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29508ae-19e9-48e1-90fb-8f0fefc9a75d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:49.022962 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29508ae-19e9-48e1-90fb-8f0fefc9a75d None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:49.032186 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f929e573-ab0d-4360-9d96-ba612452a666 None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:49.034121 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 332/667] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Aug 30 15:46:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:46:49.034929 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29508ae-19e9-48e1-90fb-8f0fefc9a75d None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:49.071829 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af275abb-b2ee-4046-bb4a-22ff376d5170 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:49.072441 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af275abb-b2ee-4046-bb4a-22ff376d5170 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:49.072713 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af275abb-b2ee-4046-bb4a-22ff376d5170 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:49.072979 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af275abb-b2ee-4046-bb4a-22ff376d5170 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:49.127579 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:49.137238 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29508ae-19e9-48e1-90fb-8f0fefc9a75d None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:49.141052 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af275abb-b2ee-4046-bb4a-22ff376d5170 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:49.142559 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af275abb-b2ee-4046-bb4a-22ff376d5170 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:49.172208 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379f0413-9a82-4526-b8aa-bd082c1fccb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:49.172746 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379f0413-9a82-4526-b8aa-bd082c1fccb9 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:49.173022 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379f0413-9a82-4526-b8aa-bd082c1fccb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:49.173317 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379f0413-9a82-4526-b8aa-bd082c1fccb9 None admin] PATH_INFO: `/v3/projects` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:49.174012 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379f0413-9a82-4526-b8aa-bd082c1fccb9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:46:49.174314 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379f0413-9a82-4526-b8aa-bd082c1fccb9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:49.175757 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379f0413-9a82-4526-b8aa-bd082c1fccb9 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:49.187764 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 333/668] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Aug 30 15:46:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:46:49.188544 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379f0413-9a82-4526-b8aa-bd082c1fccb9 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:49.217893 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:49.250097 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 336/669] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Aug 30 15:46:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 15:46:49.250936 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af275abb-b2ee-4046-bb4a-22ff376d5170 None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:49.285602 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379f0413-9a82-4526-b8aa-bd082c1fccb9 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:49.319787 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fabd2fa-16c8-4753-92cf-137101ef1223 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:49.320396 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fabd2fa-16c8-4753-92cf-137101ef1223 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:49.320681 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fabd2fa-16c8-4753-92cf-137101ef1223 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:49.320973 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fabd2fa-16c8-4753-92cf-137101ef1223 None admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:49.321674 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fabd2fa-16c8-4753-92cf-137101ef1223 None admin] RBAC: Adding query filter params (name=service) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:46:49.321976 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fabd2fa-16c8-4753-92cf-137101ef1223 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:49.323353 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fabd2fa-16c8-4753-92cf-137101ef1223 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:49.331290 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 334/670] 10.222.0.20 () {56 vars in 1103 bytes} [Tue Aug 30 15:46:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:46:49.332073 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fabd2fa-16c8-4753-92cf-137101ef1223 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:49.348743 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:49.350136 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af275abb-b2ee-4046-bb4a-22ff376d5170 None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:49.384251 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf5f9a7-09ef-4840-8eae-caa312af73e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:49.385017 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf5f9a7-09ef-4840-8eae-caa312af73e6 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:49.385448 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf5f9a7-09ef-4840-8eae-caa312af73e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:49.385829 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf5f9a7-09ef-4840-8eae-caa312af73e6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:49.392735 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf5f9a7-09ef-4840-8eae-caa312af73e6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:49.394380 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf5f9a7-09ef-4840-8eae-caa312af73e6 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:49.405935 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application [None req-6bf5f9a7-09ef-4840-8eae-caa312af73e6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 15:46:49.405935 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:46:49.405935 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:46:49.405935 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:46:49.405935 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:46:49.405935 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:46:49.405935 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:46:49.405935 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:46:49.405935 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:46:49.405935 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:46:49.405935 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:46:49.405935 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:46:49.405935 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 15:46:49.405935 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 15:46:49.405935 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 15:46:49.406946 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 15:46:49.406946 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:46:49.406946 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:46:49.406946 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:46:49.406946 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:46:49.406946 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:46:49.406946 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:46:49.406946 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:46:49.406946 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application with Lock( Aug 30 15:46:49.406946 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:46:49.406946 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:46:49.406946 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:46:49.406946 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:46:49.406946 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:46:49.406946 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:46:49.406946 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:46:49.406946 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:46:49.407955 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 15:46:49.407955 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 15:46:49.407955 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 15:46:49.407955 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 15:46:49.407955 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 15:46:49.407955 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 15:46:49.407955 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 15:46:49.407955 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Aug 30 15:46:49.407955 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 337/671] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Aug 30 15:46:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 15:46:49.407955 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf5f9a7-09ef-4840-8eae-caa312af73e6 None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:49.439153 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:49.442925 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fabd2fa-16c8-4753-92cf-137101ef1223 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:49.476612 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4b5cf0-e8c0-4e79-99f7-c1a8ca220958 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:49.477245 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4b5cf0-e8c0-4e79-99f7-c1a8ca220958 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:49.477526 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4b5cf0-e8c0-4e79-99f7-c1a8ca220958 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:49.477792 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4b5cf0-e8c0-4e79-99f7-c1a8ca220958 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:49.478520 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4b5cf0-e8c0-4e79-99f7-c1a8ca220958 None admin] RBAC: Adding query filter params (role.id=57eb8466096346b39319536c6feeb6f7, scope.project.id=854fe71b49224dff9a4b1766a20ab266, user.id=7a1e756ae9be4a7bac1e81acf07e8f3b) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:46:49.478822 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4b5cf0-e8c0-4e79-99f7-c1a8ca220958 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:49.480287 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4b5cf0-e8c0-4e79-99f7-c1a8ca220958 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:49.486326 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 335/672] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Aug 30 15:46:49 2022] GET /identity/v3/role_assignments?user.id=7a1e756ae9be4a7bac1e81acf07e8f3b&scope.project.id=854fe71b49224dff9a4b1766a20ab266&role.id=57eb8466096346b39319536c6feeb6f7 => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:46:49.487122 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4b5cf0-e8c0-4e79-99f7-c1a8ca220958 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:49.512850 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf5f9a7-09ef-4840-8eae-caa312af73e6 None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:49.555386 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f270a8ae-aed6-42e6-af30-d7038e010f89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:49.556186 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f270a8ae-aed6-42e6-af30-d7038e010f89 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:49.556562 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f270a8ae-aed6-42e6-af30-d7038e010f89 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:49.556932 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f270a8ae-aed6-42e6-af30-d7038e010f89 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:49.557839 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f270a8ae-aed6-42e6-af30-d7038e010f89 None admin] RBAC: Adding query filter params (role.id=57eb8466096346b39319536c6feeb6f7, scope.project.id=854fe71b49224dff9a4b1766a20ab266, user.id=0100a04821804993866ea20f159663f9) {{(pid=81512) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:46:49.558257 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f270a8ae-aed6-42e6-af30-d7038e010f89 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:49.559833 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f270a8ae-aed6-42e6-af30-d7038e010f89 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:49.566898 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 338/673] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Aug 30 15:46:49 2022] GET /identity/v3/role_assignments?user.id=0100a04821804993866ea20f159663f9&scope.project.id=854fe71b49224dff9a4b1766a20ab266&role.id=57eb8466096346b39319536c6feeb6f7 => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:46:49.567922 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f270a8ae-aed6-42e6-af30-d7038e010f89 None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:49.634401 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:49.653355 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4b5cf0-e8c0-4e79-99f7-c1a8ca220958 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:49.695996 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94138469-3402-4f52-bd01-5b0d81b5b407 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:49.696733 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94138469-3402-4f52-bd01-5b0d81b5b407 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:49.697101 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94138469-3402-4f52-bd01-5b0d81b5b407 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:49.697484 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94138469-3402-4f52-bd01-5b0d81b5b407 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:49.707161 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:49.711315 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94138469-3402-4f52-bd01-5b0d81b5b407 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:49.712904 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94138469-3402-4f52-bd01-5b0d81b5b407 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:49.719614 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f270a8ae-aed6-42e6-af30-d7038e010f89 None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:49.731774 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-94138469-3402-4f52-bd01-5b0d81b5b407 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 15:46:49.731774 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:46:49.731774 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:46:49.731774 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:46:49.731774 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:46:49.731774 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:46:49.731774 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:46:49.731774 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:46:49.731774 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:46:49.731774 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:46:49.731774 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:46:49.731774 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:46:49.731774 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 15:46:49.731774 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 15:46:49.731774 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 15:46:49.731774 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 15:46:49.732788 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:46:49.732788 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:46:49.732788 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:46:49.732788 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:46:49.732788 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:46:49.732788 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:46:49.732788 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:46:49.732788 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Aug 30 15:46:49.732788 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:46:49.732788 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:46:49.732788 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:46:49.732788 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:46:49.732788 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:46:49.732788 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:46:49.732788 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:46:49.732788 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:46:49.732788 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 15:46:49.733790 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 15:46:49.733790 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 15:46:49.733790 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Aug 30 15:46:49.738266 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 336/674] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 30 15:46:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 15:46:49.738266 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94138469-3402-4f52-bd01-5b0d81b5b407 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:49.772834 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eccd2f8-6a9b-4e46-ae75-91e8118b46ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:49.774561 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eccd2f8-6a9b-4e46-ae75-91e8118b46ff None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:49.774561 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eccd2f8-6a9b-4e46-ae75-91e8118b46ff None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:49.774561 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eccd2f8-6a9b-4e46-ae75-91e8118b46ff None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:49.792519 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eccd2f8-6a9b-4e46-ae75-91e8118b46ff None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:49.794217 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eccd2f8-6a9b-4e46-ae75-91e8118b46ff None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:49.811554 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application [None req-5eccd2f8-6a9b-4e46-ae75-91e8118b46ff None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 15:46:49.811554 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:46:49.811554 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:46:49.811554 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:46:49.811554 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:46:49.811554 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:46:49.811554 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:46:49.811554 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:46:49.811554 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:46:49.811554 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:46:49.811554 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:46:49.811554 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:46:49.811554 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 15:46:49.811554 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 15:46:49.811554 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 15:46:49.811554 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 15:46:49.812746 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:46:49.812746 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:46:49.812746 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:46:49.812746 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:46:49.812746 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:46:49.812746 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:46:49.812746 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:46:49.812746 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application with Lock( Aug 30 15:46:49.812746 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:46:49.812746 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:46:49.812746 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:46:49.812746 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:46:49.812746 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:46:49.812746 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:46:49.812746 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:46:49.812746 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:46:49.812746 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 15:46:49.813842 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 15:46:49.813842 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 15:46:49.813842 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Aug 30 15:46:49.813842 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 339/675] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 30 15:46:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 15:46:49.814075 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eccd2f8-6a9b-4e46-ae75-91e8118b46ff None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:49.874480 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94138469-3402-4f52-bd01-5b0d81b5b407 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:49.905578 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:49.910989 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a777a2a-89b5-4861-89c3-3d76a2c9103a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:49.911771 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a777a2a-89b5-4861-89c3-3d76a2c9103a None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:49.912281 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a777a2a-89b5-4861-89c3-3d76a2c9103a None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:49.912618 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a777a2a-89b5-4861-89c3-3d76a2c9103a None admin] PATH_INFO: `/v3/projects` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:49.913589 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a777a2a-89b5-4861-89c3-3d76a2c9103a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:46:49.913995 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a777a2a-89b5-4861-89c3-3d76a2c9103a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:49.915662 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a777a2a-89b5-4861-89c3-3d76a2c9103a None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:49.918820 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eccd2f8-6a9b-4e46-ae75-91e8118b46ff None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:49.929816 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 337/676] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Aug 30 15:46:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:46:49.930868 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a777a2a-89b5-4861-89c3-3d76a2c9103a None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:49.954821 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f6ba3c-553e-4d04-9070-44d6d4fb2e4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:49.955385 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f6ba3c-553e-4d04-9070-44d6d4fb2e4b None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:49.955667 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f6ba3c-553e-4d04-9070-44d6d4fb2e4b None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:49.955937 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f6ba3c-553e-4d04-9070-44d6d4fb2e4b None admin] PATH_INFO: `/v3/domains` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:49.956589 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f6ba3c-553e-4d04-9070-44d6d4fb2e4b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81512) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:46:49.956900 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f6ba3c-553e-4d04-9070-44d6d4fb2e4b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:49.958498 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f6ba3c-553e-4d04-9070-44d6d4fb2e4b None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:49.971667 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 340/677] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Aug 30 15:46:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:46:49.972573 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f6ba3c-553e-4d04-9070-44d6d4fb2e4b None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:49.981246 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:50.054576 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a777a2a-89b5-4861-89c3-3d76a2c9103a None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:50.075165 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f6ba3c-553e-4d04-9070-44d6d4fb2e4b None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:50.092674 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eea7974-3f5f-4106-b56f-04b4296a3808 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:50.093632 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eea7974-3f5f-4106-b56f-04b4296a3808 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:50.094082 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eea7974-3f5f-4106-b56f-04b4296a3808 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:50.094594 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eea7974-3f5f-4106-b56f-04b4296a3808 None admin] PATH_INFO: `/v3/domains` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:50.095396 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eea7974-3f5f-4106-b56f-04b4296a3808 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:46:50.095841 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eea7974-3f5f-4106-b56f-04b4296a3808 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:50.097397 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eea7974-3f5f-4106-b56f-04b4296a3808 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:50.114996 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f0dbe2-c591-4c49-80bf-d5ecc2ced5ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:50.115450 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 338/678] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Aug 30 15:46:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:46:50.115613 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f0dbe2-c591-4c49-80bf-d5ecc2ced5ce None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:50.115893 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f0dbe2-c591-4c49-80bf-d5ecc2ced5ce None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:50.116159 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f0dbe2-c591-4c49-80bf-d5ecc2ced5ce None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:50.116896 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f0dbe2-c591-4c49-80bf-d5ecc2ced5ce None admin] RBAC: Adding query filter params (role.id=57eb8466096346b39319536c6feeb6f7, scope.project.id=854fe71b49224dff9a4b1766a20ab266, user.id=b70e06a7d06d4199ab4fc36342ee1d68) {{(pid=81512) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:46:50.117130 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eea7974-3f5f-4106-b56f-04b4296a3808 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:50.117260 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f0dbe2-c591-4c49-80bf-d5ecc2ced5ce None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:50.118674 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f0dbe2-c591-4c49-80bf-d5ecc2ced5ce None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:50.126568 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 341/679] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Aug 30 15:46:49 2022] GET /identity/v3/role_assignments?user.id=b70e06a7d06d4199ab4fc36342ee1d68&scope.project.id=854fe71b49224dff9a4b1766a20ab266&role.id=57eb8466096346b39319536c6feeb6f7 => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:46:50.127409 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f0dbe2-c591-4c49-80bf-d5ecc2ced5ce None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:50.202574 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:50.263146 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:50.296989 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eea7974-3f5f-4106-b56f-04b4296a3808 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:50.327503 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f0dbe2-c591-4c49-80bf-d5ecc2ced5ce None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:50.345384 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f37ed7c-ccef-434c-9038-f0d0b9ac527f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:50.346286 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f37ed7c-ccef-434c-9038-f0d0b9ac527f None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:50.346707 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f37ed7c-ccef-434c-9038-f0d0b9ac527f None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:50.347127 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f37ed7c-ccef-434c-9038-f0d0b9ac527f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:50.359422 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f37ed7c-ccef-434c-9038-f0d0b9ac527f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:50.363369 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f37ed7c-ccef-434c-9038-f0d0b9ac527f None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:50.371552 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30234a0b-9dc7-47c1-b1f5-1d9e5ddefd36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:50.371724 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30234a0b-9dc7-47c1-b1f5-1d9e5ddefd36 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:50.371939 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30234a0b-9dc7-47c1-b1f5-1d9e5ddefd36 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:50.372180 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30234a0b-9dc7-47c1-b1f5-1d9e5ddefd36 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:50.378790 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-6f37ed7c-ccef-434c-9038-f0d0b9ac527f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 15:46:50.378790 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:46:50.378790 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:46:50.378790 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:46:50.378790 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:46:50.378790 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:46:50.378790 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:46:50.378790 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:46:50.378790 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:46:50.378790 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:46:50.378790 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:46:50.378790 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:46:50.378790 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 15:46:50.378790 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 15:46:50.378790 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 15:46:50.379809 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 15:46:50.379809 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:46:50.379809 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:46:50.379809 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:46:50.379809 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:46:50.379809 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:46:50.379809 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:46:50.379809 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:46:50.379809 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Aug 30 15:46:50.379809 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:46:50.379809 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:46:50.379809 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:46:50.379809 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:46:50.379809 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:46:50.379809 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:46:50.379809 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:46:50.379809 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:46:50.380799 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 15:46:50.380799 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 15:46:50.380799 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 15:46:50.380799 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 15:46:50.380799 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 15:46:50.380799 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 15:46:50.380799 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 15:46:50.380799 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Aug 30 15:46:50.381292 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30234a0b-9dc7-47c1-b1f5-1d9e5ddefd36 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:50.383038 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 339/680] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Aug 30 15:46:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 15:46:50.383324 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30234a0b-9dc7-47c1-b1f5-1d9e5ddefd36 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:50.391762 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f37ed7c-ccef-434c-9038-f0d0b9ac527f None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:50.398386 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application [None req-30234a0b-9dc7-47c1-b1f5-1d9e5ddefd36 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 30 15:46:50.398386 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:46:50.398386 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:46:50.398386 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:46:50.398386 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:46:50.398386 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:46:50.398386 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:46:50.398386 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:46:50.398386 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:46:50.398386 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:46:50.398386 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:46:50.398386 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:46:50.398386 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 15:46:50.398386 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 15:46:50.398386 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 15:46:50.398386 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 15:46:50.399414 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:46:50.399414 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:46:50.399414 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 15:46:50.399414 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 15:46:50.399414 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 15:46:50.399414 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 15:46:50.399414 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:46:50.399414 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:46:50.399414 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:46:50.399414 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:46:50.399414 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:46:50.399414 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application with Lock( Aug 30 15:46:50.399414 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:46:50.399414 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:46:50.399414 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:46:50.399414 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:46:50.399414 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:46:50.399414 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:46:50.400382 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:46:50.400382 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:46:50.400382 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 15:46:50.400382 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 15:46:50.400382 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 15:46:50.400382 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 15:46:50.400382 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 15:46:50.400382 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 15:46:50.400382 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 30 15:46:50.400382 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Aug 30 15:46:50.400382 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 342/681] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Aug 30 15:46:50 2022] GET /identity/v3/users/swift => generated 83 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 15:46:50.405639 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30234a0b-9dc7-47c1-b1f5-1d9e5ddefd36 None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:50.520083 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f37ed7c-ccef-434c-9038-f0d0b9ac527f None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:50.531870 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:50.546979 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30234a0b-9dc7-47c1-b1f5-1d9e5ddefd36 None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:50.561739 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19782226-0469-4cf9-b20f-f0286e389282 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:50.562710 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19782226-0469-4cf9-b20f-f0286e389282 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:50.563206 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19782226-0469-4cf9-b20f-f0286e389282 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:50.563672 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19782226-0469-4cf9-b20f-f0286e389282 None admin] PATH_INFO: `/v3/projects` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:50.564577 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19782226-0469-4cf9-b20f-f0286e389282 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:46:50.565078 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19782226-0469-4cf9-b20f-f0286e389282 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:50.566765 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19782226-0469-4cf9-b20f-f0286e389282 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:50.580203 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 340/682] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Aug 30 15:46:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 15:46:50.582311 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:50.586751 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19782226-0469-4cf9-b20f-f0286e389282 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:50.589518 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ae2507-431c-44d7-aaf5-6d67b444c45a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:50.590903 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ae2507-431c-44d7-aaf5-6d67b444c45a None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:50.592556 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ae2507-431c-44d7-aaf5-6d67b444c45a None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:50.592835 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ae2507-431c-44d7-aaf5-6d67b444c45a None admin] PATH_INFO: `/v3/users` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:50.593697 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ae2507-431c-44d7-aaf5-6d67b444c45a None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81512) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:46:50.594013 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ae2507-431c-44d7-aaf5-6d67b444c45a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:50.595461 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ae2507-431c-44d7-aaf5-6d67b444c45a None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:50.618080 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 343/683] 10.222.0.20 () {56 vars in 1135 bytes} [Tue Aug 30 15:46:50 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:46:50.624364 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ae2507-431c-44d7-aaf5-6d67b444c45a None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:50.709776 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19782226-0469-4cf9-b20f-f0286e389282 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:50.762134 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ae2507-431c-44d7-aaf5-6d67b444c45a None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:50.765161 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41b3172-98a0-4102-a433-535637615bb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:50.765935 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41b3172-98a0-4102-a433-535637615bb3 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:50.766304 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41b3172-98a0-4102-a433-535637615bb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:50.766837 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41b3172-98a0-4102-a433-535637615bb3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:50.768269 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41b3172-98a0-4102-a433-535637615bb3 None admin] RBAC: Adding query filter params (role.id=57eb8466096346b39319536c6feeb6f7, scope.project.id=854fe71b49224dff9a4b1766a20ab266, user.id=65f4157105f3488f807b7475008ab722) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:46:50.768870 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41b3172-98a0-4102-a433-535637615bb3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:50.770521 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41b3172-98a0-4102-a433-535637615bb3 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:50.785983 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 341/684] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Aug 30 15:46:50 2022] GET /identity/v3/role_assignments?user.id=65f4157105f3488f807b7475008ab722&scope.project.id=854fe71b49224dff9a4b1766a20ab266&role.id=57eb8466096346b39319536c6feeb6f7 => generated 608 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 15:46:50.825645 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626890c9-5e25-451e-b84f-6c21d9d3b102 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:50.826594 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626890c9-5e25-451e-b84f-6c21d9d3b102 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:50.827023 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626890c9-5e25-451e-b84f-6c21d9d3b102 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:50.827408 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626890c9-5e25-451e-b84f-6c21d9d3b102 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:50.909594 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:50.931865 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626890c9-5e25-451e-b84f-6c21d9d3b102 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:50.933585 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626890c9-5e25-451e-b84f-6c21d9d3b102 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:51.047114 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 344/685] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Aug 30 15:46:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 15:46:51.052292 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41b3172-98a0-4102-a433-535637615bb3 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:51.097638 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:51.190891 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41b3172-98a0-4102-a433-535637615bb3 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:51.234964 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee34d5fa-2d3d-459a-b1aa-786b6cc44a4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:51.236004 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee34d5fa-2d3d-459a-b1aa-786b6cc44a4c None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:51.236480 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee34d5fa-2d3d-459a-b1aa-786b6cc44a4c None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:51.236920 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee34d5fa-2d3d-459a-b1aa-786b6cc44a4c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:51.247835 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee34d5fa-2d3d-459a-b1aa-786b6cc44a4c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:51.249934 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee34d5fa-2d3d-459a-b1aa-786b6cc44a4c None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:51.269179 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-ee34d5fa-2d3d-459a-b1aa-786b6cc44a4c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 15:46:51.269179 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:46:51.269179 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:46:51.269179 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:46:51.269179 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:46:51.269179 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:46:51.269179 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:46:51.269179 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:46:51.269179 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:46:51.269179 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:46:51.269179 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:46:51.269179 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:46:51.269179 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 15:46:51.269179 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 15:46:51.269179 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 15:46:51.269179 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 15:46:51.270217 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:46:51.270217 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:46:51.270217 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:46:51.270217 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:46:51.270217 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:46:51.270217 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:46:51.270217 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:46:51.270217 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Aug 30 15:46:51.270217 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:46:51.270217 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:46:51.270217 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:46:51.270217 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:46:51.270217 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:46:51.270217 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:46:51.270217 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:46:51.270217 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:46:51.270217 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 15:46:51.271122 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 15:46:51.271122 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 15:46:51.271122 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Aug 30 15:46:51.273031 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 342/686] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 30 15:46:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 15:46:51.278348 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626890c9-5e25-451e-b84f-6c21d9d3b102 None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:51.398504 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:51.410794 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626890c9-5e25-451e-b84f-6c21d9d3b102 None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:51.445380 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91672800-877b-47be-98e1-1708b0b210d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:51.445995 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91672800-877b-47be-98e1-1708b0b210d8 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:51.446269 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91672800-877b-47be-98e1-1708b0b210d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:51.446539 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91672800-877b-47be-98e1-1708b0b210d8 None admin] PATH_INFO: `/v3/domains` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:51.447182 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91672800-877b-47be-98e1-1708b0b210d8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81512) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:46:51.447481 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91672800-877b-47be-98e1-1708b0b210d8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:51.448927 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91672800-877b-47be-98e1-1708b0b210d8 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:51.460720 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 345/687] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Aug 30 15:46:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 15:46:51.465943 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee34d5fa-2d3d-459a-b1aa-786b6cc44a4c None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:51.581725 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:51.597703 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee34d5fa-2d3d-459a-b1aa-786b6cc44a4c None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:51.637253 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12f744c-d594-4e57-ad84-ca70ac72b7fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:51.637253 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12f744c-d594-4e57-ad84-ca70ac72b7fb None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:51.637253 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12f744c-d594-4e57-ad84-ca70ac72b7fb None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:51.637253 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12f744c-d594-4e57-ad84-ca70ac72b7fb None admin] PATH_INFO: `/v3/projects/service` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:51.639595 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12f744c-d594-4e57-ad84-ca70ac72b7fb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:51.642942 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12f744c-d594-4e57-ad84-ca70ac72b7fb None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:51.655711 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-c12f744c-d594-4e57-ad84-ca70ac72b7fb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 15:46:51.655711 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:46:51.655711 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:46:51.655711 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:46:51.655711 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:46:51.655711 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:46:51.655711 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:46:51.655711 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:46:51.655711 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:46:51.655711 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:46:51.655711 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:46:51.655711 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:46:51.655711 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 15:46:51.655711 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 15:46:51.655711 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 15:46:51.656629 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 15:46:51.656629 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:46:51.656629 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:46:51.656629 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:46:51.656629 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:46:51.656629 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:46:51.656629 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:46:51.656629 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:46:51.656629 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Aug 30 15:46:51.656629 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:46:51.656629 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:46:51.656629 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:46:51.656629 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:46:51.656629 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:46:51.656629 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:46:51.656629 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:46:51.656629 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:46:51.657568 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 15:46:51.657568 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 15:46:51.657568 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 15:46:51.657568 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 15:46:51.657568 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 15:46:51.657568 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 15:46:51.657568 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 15:46:51.657568 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Aug 30 15:46:51.657568 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 343/688] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Aug 30 15:46:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 15:46:51.663599 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91672800-877b-47be-98e1-1708b0b210d8 None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:51.666001 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70888204-8838-47cb-8e5b-3c45873d17e3 None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:51.666286 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70888204-8838-47cb-8e5b-3c45873d17e3 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:51.666560 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70888204-8838-47cb-8e5b-3c45873d17e3 None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:51.668397 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 344/689] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 30 15:46:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 15:46:51.673702 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e02854e-efd9-470d-b7b3-8d823964b238 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:51.673980 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e02854e-efd9-470d-b7b3-8d823964b238 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:51.674252 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e02854e-efd9-470d-b7b3-8d823964b238 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:51.694419 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-7e02854e-efd9-470d-b7b3-8d823964b238 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:46:51.695994 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-7e02854e-efd9-470d-b7b3-8d823964b238 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:46:51.808623 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91672800-877b-47be-98e1-1708b0b210d8 None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:51.814839 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 345/690] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 30 15:46:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 15:46:51.858515 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdce3503-7659-47ed-94d6-fdbd7c5170a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:51.859141 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdce3503-7659-47ed-94d6-fdbd7c5170a2 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:51.859422 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdce3503-7659-47ed-94d6-fdbd7c5170a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:51.859685 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdce3503-7659-47ed-94d6-fdbd7c5170a2 None admin] PATH_INFO: `/v3/projects` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:51.860396 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdce3503-7659-47ed-94d6-fdbd7c5170a2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81512) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:46:51.860703 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdce3503-7659-47ed-94d6-fdbd7c5170a2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:51.866261 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdce3503-7659-47ed-94d6-fdbd7c5170a2 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:51.880345 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 346/691] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Aug 30 15:46:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:46:51.886082 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e02854e-efd9-470d-b7b3-8d823964b238 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:51.901102 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fb3d63-9464-4e5b-ac26-8b70ad1150c2 None None] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:51.901439 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fb3d63-9464-4e5b-ac26-8b70ad1150c2 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:51.901691 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fb3d63-9464-4e5b-ac26-8b70ad1150c2 None None] PATH_INFO: `/` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:51.903163 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 347/692] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 30 15:46:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 15:46:51.910652 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a423f8-ff47-451c-b3a1-3a7a9f476f8f None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:51.910868 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a423f8-ff47-451c-b3a1-3a7a9f476f8f None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:51.911125 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a423f8-ff47-451c-b3a1-3a7a9f476f8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:51.929257 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-73a423f8-ff47-451c-b3a1-3a7a9f476f8f None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:46:51.930468 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-73a423f8-ff47-451c-b3a1-3a7a9f476f8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:46:51.932465 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:52.033580 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:52.038456 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 348/693] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 30 15:46:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 15:46:52.055472 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e02854e-efd9-470d-b7b3-8d823964b238 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:52.089051 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5a840d-584b-4ecb-abae-c1a5b4a6f962 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:52.089685 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5a840d-584b-4ecb-abae-c1a5b4a6f962 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:52.089963 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5a840d-584b-4ecb-abae-c1a5b4a6f962 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:52.090229 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5a840d-584b-4ecb-abae-c1a5b4a6f962 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:52.090959 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5a840d-584b-4ecb-abae-c1a5b4a6f962 None admin] RBAC: Adding query filter params (role.id=57eb8466096346b39319536c6feeb6f7, scope.project.id=854fe71b49224dff9a4b1766a20ab266, user.id=8fdafdc39a594bb4b0aeaf2fbc84c2fb) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:46:52.091268 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5a840d-584b-4ecb-abae-c1a5b4a6f962 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:52.092735 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5a840d-584b-4ecb-abae-c1a5b4a6f962 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:52.098914 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 346/694] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Aug 30 15:46:51 2022] GET /identity/v3/role_assignments?user.id=8fdafdc39a594bb4b0aeaf2fbc84c2fb&scope.project.id=854fe71b49224dff9a4b1766a20ab266&role.id=57eb8466096346b39319536c6feeb6f7 => generated 608 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:46:52.145558 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a423f8-ff47-451c-b3a1-3a7a9f476f8f None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:52.283570 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a423f8-ff47-451c-b3a1-3a7a9f476f8f None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:52.322276 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddc1680-af00-4a92-8ebe-81456f5b72b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:52.322911 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddc1680-af00-4a92-8ebe-81456f5b72b6 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:52.323190 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddc1680-af00-4a92-8ebe-81456f5b72b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:52.323461 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddc1680-af00-4a92-8ebe-81456f5b72b6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:52.330624 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:52.335689 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fddc1680-af00-4a92-8ebe-81456f5b72b6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 15:46:52.340490 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application [None req-fddc1680-af00-4a92-8ebe-81456f5b72b6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 15:46:52.340490 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:46:52.340490 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:46:52.340490 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:46:52.340490 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:46:52.340490 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:46:52.340490 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:46:52.340490 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:46:52.340490 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:46:52.340490 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:46:52.340490 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:46:52.340490 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:46:52.340490 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 15:46:52.340490 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 15:46:52.340490 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 15:46:52.340490 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 15:46:52.341469 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 15:46:52.341469 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 15:46:52.341469 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 15:46:52.341469 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 15:46:52.341469 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:46:52.341469 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:46:52.341469 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:46:52.341469 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:46:52.341469 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:46:52.341469 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:46:52.341469 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:46:52.341469 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application with Lock( Aug 30 15:46:52.341469 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:46:52.341469 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:46:52.341469 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:46:52.341469 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:46:52.342553 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:46:52.342553 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:46:52.342553 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:46:52.342553 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:46:52.342553 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 15:46:52.342553 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 15:46:52.342553 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 15:46:52.342553 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 15:46:52.342553 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 15:46:52.342553 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 15:46:52.342553 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 15:46:52.342553 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Aug 30 15:46:52.342553 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 349/695] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Aug 30 15:46:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 15:46:52.348061 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5a840d-584b-4ecb-abae-c1a5b4a6f962 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:52.385380 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddc1680-af00-4a92-8ebe-81456f5b72b6 None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:52.473882 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5a840d-584b-4ecb-abae-c1a5b4a6f962 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:52.487408 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddc1680-af00-4a92-8ebe-81456f5b72b6 None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:52.518262 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394ab46a-f2de-4a0a-8ae4-7d25f34cdd98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:52.519060 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394ab46a-f2de-4a0a-8ae4-7d25f34cdd98 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:52.519416 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394ab46a-f2de-4a0a-8ae4-7d25f34cdd98 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:52.519749 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394ab46a-f2de-4a0a-8ae4-7d25f34cdd98 None admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:52.520600 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394ab46a-f2de-4a0a-8ae4-7d25f34cdd98 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:46:52.521033 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394ab46a-f2de-4a0a-8ae4-7d25f34cdd98 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:52.522555 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394ab46a-f2de-4a0a-8ae4-7d25f34cdd98 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:52.528751 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d370afb1-3027-47a4-a175-d36eea87a096 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:52.529527 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:52.529902 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d370afb1-3027-47a4-a175-d36eea87a096 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:52.530372 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d370afb1-3027-47a4-a175-d36eea87a096 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:52.530826 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d370afb1-3027-47a4-a175-d36eea87a096 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:52.531598 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d370afb1-3027-47a4-a175-d36eea87a096 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:52.533386 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d370afb1-3027-47a4-a175-d36eea87a096 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:52.534980 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 347/696] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Aug 30 15:46:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:46:52.541055 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394ab46a-f2de-4a0a-8ae4-7d25f34cdd98 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:52.547143 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application [None req-d370afb1-3027-47a4-a175-d36eea87a096 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 30 15:46:52.547143 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:46:52.547143 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 15:46:52.547143 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 15:46:52.547143 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 15:46:52.547143 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 15:46:52.547143 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 15:46:52.547143 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 15:46:52.547143 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 30 15:46:52.547143 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Aug 30 15:46:52.547143 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 15:46:52.547143 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Aug 30 15:46:52.547143 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:46:52.547143 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:46:52.547143 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:46:52.547143 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:46:52.547143 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:46:52.548168 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:46:52.548168 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:46:52.548168 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:46:52.548168 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:46:52.548168 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:46:52.548168 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:46:52.548168 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 15:46:52.548168 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 15:46:52.548168 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 15:46:52.548168 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 15:46:52.548168 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:46:52.548168 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:46:52.548168 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:46:52.548168 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:46:52.548168 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:46:52.548168 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:46:52.549001 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:46:52.549001 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application with Lock( Aug 30 15:46:52.549001 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:46:52.549001 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:46:52.549001 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:46:52.549001 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:46:52.549001 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:46:52.549001 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:46:52.549001 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:46:52.549001 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:46:52.549001 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 15:46:52.549001 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 15:46:52.549001 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 30 15:46:52.549001 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Aug 30 15:46:52.556385 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 350/697] 10.222.0.20 () {56 vars in 1129 bytes} [Tue Aug 30 15:46:52 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 15:46:52.557883 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d370afb1-3027-47a4-a175-d36eea87a096 None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:52.627804 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:52.694952 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d370afb1-3027-47a4-a175-d36eea87a096 None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:52.745244 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b15c227-7735-4d95-a859-fc1e7f2ac5c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:52.745244 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b15c227-7735-4d95-a859-fc1e7f2ac5c7 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:52.745912 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b15c227-7735-4d95-a859-fc1e7f2ac5c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:52.749244 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b15c227-7735-4d95-a859-fc1e7f2ac5c7 None admin] PATH_INFO: `/v3/services` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:52.749244 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b15c227-7735-4d95-a859-fc1e7f2ac5c7 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81512) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:46:52.749244 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b15c227-7735-4d95-a859-fc1e7f2ac5c7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:52.749244 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b15c227-7735-4d95-a859-fc1e7f2ac5c7 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:52.759889 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 351/698] 10.222.0.20 () {56 vars in 1124 bytes} [Tue Aug 30 15:46:52 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 15:46:52.765253 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b15c227-7735-4d95-a859-fc1e7f2ac5c7 None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:52.847511 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394ab46a-f2de-4a0a-8ae4-7d25f34cdd98 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:52.865259 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b15c227-7735-4d95-a859-fc1e7f2ac5c7 None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:52.921246 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:52.929240 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f23499f-89c3-40b9-b220-a14f8361cf94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:52.929240 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f23499f-89c3-40b9-b220-a14f8361cf94 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:52.929240 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f23499f-89c3-40b9-b220-a14f8361cf94 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:52.929240 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f23499f-89c3-40b9-b220-a14f8361cf94 None admin] PATH_INFO: `/v3/services` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:52.929637 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f23499f-89c3-40b9-b220-a14f8361cf94 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81512) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:46:52.929637 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f23499f-89c3-40b9-b220-a14f8361cf94 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:52.929637 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f23499f-89c3-40b9-b220-a14f8361cf94 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:52.944469 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 352/699] 10.222.0.20 () {56 vars in 1124 bytes} [Tue Aug 30 15:46:52 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 15:46:52.979092 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364b1bbf-1357-431e-9de8-7f67628b328c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:52.979761 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364b1bbf-1357-431e-9de8-7f67628b328c None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:52.980047 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364b1bbf-1357-431e-9de8-7f67628b328c None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:52.980318 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364b1bbf-1357-431e-9de8-7f67628b328c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:53.001029 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364b1bbf-1357-431e-9de8-7f67628b328c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:53.002553 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364b1bbf-1357-431e-9de8-7f67628b328c None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:53.018512 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-364b1bbf-1357-431e-9de8-7f67628b328c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 15:46:53.018512 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:46:53.018512 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:46:53.018512 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:46:53.018512 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:46:53.018512 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:46:53.018512 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:46:53.018512 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:46:53.018512 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:46:53.018512 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:46:53.018512 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:46:53.018512 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:46:53.018512 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 15:46:53.018512 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 15:46:53.018512 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 15:46:53.018512 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 15:46:53.019614 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:46:53.019614 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:46:53.019614 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:46:53.019614 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:46:53.019614 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:46:53.019614 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:46:53.019614 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:46:53.019614 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Aug 30 15:46:53.019614 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:46:53.019614 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:46:53.019614 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:46:53.019614 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:46:53.019614 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:46:53.019614 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:46:53.019614 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:46:53.019614 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:46:53.019614 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 15:46:53.020583 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 15:46:53.020583 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 15:46:53.020583 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Aug 30 15:46:53.020583 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 348/700] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 30 15:46:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 479 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 15:46:53.026652 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f23499f-89c3-40b9-b220-a14f8361cf94 None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:53.127592 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d3c32b-318d-4ce8-8533-6d67d031f184 None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:53.127895 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d3c32b-318d-4ce8-8533-6d67d031f184 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:53.128748 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d3c32b-318d-4ce8-8533-6d67d031f184 None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:53.133654 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 349/701] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 30 15:46:53 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 15:46:53.135057 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b652a1-b0cc-4926-a9b0-9eff6593403e None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:53.135343 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b652a1-b0cc-4926-a9b0-9eff6593403e None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:53.135619 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b652a1-b0cc-4926-a9b0-9eff6593403e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:53.182456 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-04b652a1-b0cc-4926-a9b0-9eff6593403e None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:46:53.183696 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-04b652a1-b0cc-4926-a9b0-9eff6593403e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:46:53.187915 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f23499f-89c3-40b9-b220-a14f8361cf94 None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:53.233966 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ccf1eb-1e54-4955-a9f0-6598e3f13f12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:53.237243 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ccf1eb-1e54-4955-a9f0-6598e3f13f12 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:53.237243 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ccf1eb-1e54-4955-a9f0-6598e3f13f12 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:53.237243 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ccf1eb-1e54-4955-a9f0-6598e3f13f12 None admin] PATH_INFO: `/v3/domains` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:53.241240 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ccf1eb-1e54-4955-a9f0-6598e3f13f12 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81512) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:46:53.241240 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ccf1eb-1e54-4955-a9f0-6598e3f13f12 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:53.245245 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ccf1eb-1e54-4955-a9f0-6598e3f13f12 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:53.273110 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 353/702] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Aug 30 15:46:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:46:53.274411 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ccf1eb-1e54-4955-a9f0-6598e3f13f12 None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:53.302352 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:53.342705 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 350/703] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 30 15:46:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 15:46:53.369394 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:53.414395 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ccf1eb-1e54-4955-a9f0-6598e3f13f12 None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:53.450291 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55634e6-2fad-4a5e-9003-a4a6a6ee1d8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:53.450921 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55634e6-2fad-4a5e-9003-a4a6a6ee1d8c None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:53.451203 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55634e6-2fad-4a5e-9003-a4a6a6ee1d8c None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:53.451474 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55634e6-2fad-4a5e-9003-a4a6a6ee1d8c None admin] PATH_INFO: `/v3/users/nova` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:53.459118 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55634e6-2fad-4a5e-9003-a4a6a6ee1d8c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:53.460578 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55634e6-2fad-4a5e-9003-a4a6a6ee1d8c None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:53.472374 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application [None req-c55634e6-2fad-4a5e-9003-a4a6a6ee1d8c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 30 15:46:53.472374 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:46:53.472374 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:46:53.472374 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:46:53.472374 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:46:53.472374 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:46:53.472374 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:46:53.472374 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:46:53.472374 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:46:53.472374 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:46:53.472374 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:46:53.472374 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:46:53.472374 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 15:46:53.472374 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 15:46:53.472374 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 15:46:53.472374 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 15:46:53.474613 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:46:53.474613 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:46:53.474613 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 15:46:53.474613 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 15:46:53.474613 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 15:46:53.474613 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 15:46:53.474613 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:46:53.474613 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:46:53.474613 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:46:53.474613 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:46:53.474613 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:46:53.474613 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application with Lock( Aug 30 15:46:53.474613 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:46:53.474613 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:46:53.474613 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:46:53.474613 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:46:53.474613 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:46:53.474613 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:46:53.476714 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:46:53.476714 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:46:53.476714 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 15:46:53.476714 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 15:46:53.476714 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 15:46:53.476714 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 15:46:53.476714 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 15:46:53.476714 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 15:46:53.476714 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 30 15:46:53.476714 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Aug 30 15:46:53.476714 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 354/704] 10.222.0.20 () {56 vars in 1093 bytes} [Tue Aug 30 15:46:53 2022] GET /identity/v3/users/nova => generated 82 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 15:46:53.486511 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b652a1-b0cc-4926-a9b0-9eff6593403e None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:53.532759 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb0dd4d-dd93-4b43-97b2-adce8dd49993 None None] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:53.533085 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb0dd4d-dd93-4b43-97b2-adce8dd49993 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:53.533329 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb0dd4d-dd93-4b43-97b2-adce8dd49993 None None] PATH_INFO: `/` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:53.535027 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 355/705] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 30 15:46:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 15:46:53.541325 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191f5564-42fd-4f3c-a8c4-557d1598a947 None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:53.541605 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191f5564-42fd-4f3c-a8c4-557d1598a947 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:53.541876 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191f5564-42fd-4f3c-a8c4-557d1598a947 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:53.562418 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-191f5564-42fd-4f3c-a8c4-557d1598a947 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:46:53.564100 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-191f5564-42fd-4f3c-a8c4-557d1598a947 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:46:53.645397 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 356/706] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 30 15:46:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 15:46:53.669114 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191f5564-42fd-4f3c-a8c4-557d1598a947 None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:53.707996 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b652a1-b0cc-4926-a9b0-9eff6593403e None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:53.721592 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:53.780135 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191f5564-42fd-4f3c-a8c4-557d1598a947 None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:53.794504 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1038cfd7-cec7-4d2a-9ec3-b3e57b6c9c1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:53.795136 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1038cfd7-cec7-4d2a-9ec3-b3e57b6c9c1e None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:53.795620 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1038cfd7-cec7-4d2a-9ec3-b3e57b6c9c1e None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:53.795903 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1038cfd7-cec7-4d2a-9ec3-b3e57b6c9c1e None admin] PATH_INFO: `/v3/users` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:53.796776 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1038cfd7-cec7-4d2a-9ec3-b3e57b6c9c1e None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:46:53.797087 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1038cfd7-cec7-4d2a-9ec3-b3e57b6c9c1e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:53.798634 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1038cfd7-cec7-4d2a-9ec3-b3e57b6c9c1e None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:53.824508 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f89043-13cd-44ec-a593-78d28d9d8a49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:53.825147 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f89043-13cd-44ec-a593-78d28d9d8a49 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:53.825453 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f89043-13cd-44ec-a593-78d28d9d8a49 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:53.825719 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f89043-13cd-44ec-a593-78d28d9d8a49 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:53.839327 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 351/707] 10.222.0.20 () {56 vars in 1133 bytes} [Tue Aug 30 15:46:53 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 353 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:46:53.842601 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f89043-13cd-44ec-a593-78d28d9d8a49 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:53.844160 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f89043-13cd-44ec-a593-78d28d9d8a49 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:53.845161 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1038cfd7-cec7-4d2a-9ec3-b3e57b6c9c1e None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:53.873754 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application [None req-a3f89043-13cd-44ec-a593-78d28d9d8a49 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 15:46:53.873754 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:46:53.873754 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:46:53.873754 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:46:53.873754 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:46:53.873754 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:46:53.873754 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:46:53.873754 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:46:53.873754 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:46:53.873754 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:46:53.873754 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:46:53.873754 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:46:53.873754 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 15:46:53.873754 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 15:46:53.873754 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 15:46:53.873754 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 15:46:53.875078 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:46:53.875078 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:46:53.875078 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:46:53.875078 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:46:53.875078 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:46:53.875078 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:46:53.875078 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:46:53.875078 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application with Lock( Aug 30 15:46:53.875078 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:46:53.875078 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:46:53.875078 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:46:53.875078 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:46:53.875078 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:46:53.875078 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:46:53.875078 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:46:53.875078 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:46:53.875078 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 15:46:53.875986 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 15:46:53.875986 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 15:46:53.875986 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Aug 30 15:46:53.875986 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 357/708] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 30 15:46:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 15:46:53.882093 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f89043-13cd-44ec-a593-78d28d9d8a49 None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:53.943236 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:54.009432 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1038cfd7-cec7-4d2a-9ec3-b3e57b6c9c1e None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:54.035141 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:54.047438 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f89043-13cd-44ec-a593-78d28d9d8a49 None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:54.048930 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5615b6a6-5f90-4e57-8f5b-23c6c0a9c838 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:54.049580 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5615b6a6-5f90-4e57-8f5b-23c6c0a9c838 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:54.049851 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5615b6a6-5f90-4e57-8f5b-23c6c0a9c838 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:54.050117 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5615b6a6-5f90-4e57-8f5b-23c6c0a9c838 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:54.090884 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142fbb33-ac23-4eae-825a-48df7778ed4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:54.092184 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142fbb33-ac23-4eae-825a-48df7778ed4a None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:54.092754 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142fbb33-ac23-4eae-825a-48df7778ed4a None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:54.093163 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142fbb33-ac23-4eae-825a-48df7778ed4a None admin] PATH_INFO: `/v3/domains` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:54.094316 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142fbb33-ac23-4eae-825a-48df7778ed4a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81512) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:46:54.094776 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142fbb33-ac23-4eae-825a-48df7778ed4a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:54.096813 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142fbb33-ac23-4eae-825a-48df7778ed4a None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:54.117407 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 358/709] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Aug 30 15:46:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:46:54.117407 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142fbb33-ac23-4eae-825a-48df7778ed4a None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:54.129251 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5615b6a6-5f90-4e57-8f5b-23c6c0a9c838 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:54.131415 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5615b6a6-5f90-4e57-8f5b-23c6c0a9c838 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:54.182763 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:54.228375 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142fbb33-ac23-4eae-825a-48df7778ed4a None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:54.269243 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 352/710] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Aug 30 15:46:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 30 15:46:54.269243 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5615b6a6-5f90-4e57-8f5b-23c6c0a9c838 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:54.272833 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626d88dc-4209-4fcf-8052-e695eb753a5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:54.273522 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626d88dc-4209-4fcf-8052-e695eb753a5c None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:54.273800 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626d88dc-4209-4fcf-8052-e695eb753a5c None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:54.274072 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626d88dc-4209-4fcf-8052-e695eb753a5c None admin] PATH_INFO: `/v3/services/network` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:54.274670 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626d88dc-4209-4fcf-8052-e695eb753a5c None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:54.276083 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626d88dc-4209-4fcf-8052-e695eb753a5c None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:54.285893 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application [None req-626d88dc-4209-4fcf-8052-e695eb753a5c None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 30 15:46:54.285893 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:46:54.285893 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 15:46:54.285893 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 15:46:54.285893 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 15:46:54.285893 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 15:46:54.285893 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 15:46:54.285893 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 15:46:54.285893 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 30 15:46:54.285893 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Aug 30 15:46:54.285893 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 15:46:54.285893 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Aug 30 15:46:54.285893 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:46:54.285893 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:46:54.285893 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:46:54.285893 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:46:54.285893 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:46:54.286959 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:46:54.286959 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:46:54.286959 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:46:54.286959 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:46:54.286959 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:46:54.286959 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:46:54.286959 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 15:46:54.286959 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 15:46:54.286959 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 15:46:54.286959 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 15:46:54.286959 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:46:54.286959 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:46:54.286959 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:46:54.286959 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:46:54.286959 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:46:54.286959 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:46:54.287801 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:46:54.287801 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application with Lock( Aug 30 15:46:54.287801 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:46:54.287801 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:46:54.287801 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:46:54.287801 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:46:54.287801 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:46:54.287801 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:46:54.287801 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:46:54.287801 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:46:54.287801 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 15:46:54.287801 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 15:46:54.287801 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 30 15:46:54.287801 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Aug 30 15:46:54.287801 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 359/711] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Aug 30 15:46:54 2022] GET /identity/v3/services/network => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 15:46:54.288597 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626d88dc-4209-4fcf-8052-e695eb753a5c None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:54.310688 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:54.380788 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5615b6a6-5f90-4e57-8f5b-23c6c0a9c838 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:54.397520 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626d88dc-4209-4fcf-8052-e695eb753a5c None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:54.416939 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a68975-1f70-4cec-921f-4e5b804a0e1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:54.417585 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a68975-1f70-4cec-921f-4e5b804a0e1a None admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:54.417860 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a68975-1f70-4cec-921f-4e5b804a0e1a None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:54.418228 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a68975-1f70-4cec-921f-4e5b804a0e1a None admin] PATH_INFO: `/v3/users` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:54.418883 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a68975-1f70-4cec-921f-4e5b804a0e1a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:54.420314 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a68975-1f70-4cec-921f-4e5b804a0e1a None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:54.432455 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:54.432838 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b43da5-2abd-4e65-a1ac-b7baf2f81134 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:54.433523 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b43da5-2abd-4e65-a1ac-b7baf2f81134 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:54.433917 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b43da5-2abd-4e65-a1ac-b7baf2f81134 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:54.434033 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b43da5-2abd-4e65-a1ac-b7baf2f81134 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:54.446162 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b43da5-2abd-4e65-a1ac-b7baf2f81134 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:54.447849 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b43da5-2abd-4e65-a1ac-b7baf2f81134 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:54.450575 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 353/712] 10.222.0.20 () {60 vars in 1132 bytes} [Tue Aug 30 15:46:54 2022] POST /identity/v3/users => generated 254 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 15:46:54.451954 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a68975-1f70-4cec-921f-4e5b804a0e1a None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:54.463770 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application [None req-77b43da5-2abd-4e65-a1ac-b7baf2f81134 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 15:46:54.463770 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:46:54.463770 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:46:54.463770 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:46:54.463770 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:46:54.463770 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:46:54.463770 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:46:54.463770 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:46:54.463770 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:46:54.463770 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:46:54.463770 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:46:54.463770 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:46:54.463770 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 15:46:54.463770 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 15:46:54.463770 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 15:46:54.463770 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 15:46:54.464908 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:46:54.464908 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:46:54.464908 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:46:54.464908 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:46:54.464908 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:46:54.464908 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:46:54.464908 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:46:54.464908 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application with Lock( Aug 30 15:46:54.464908 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:46:54.464908 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:46:54.464908 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:46:54.464908 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:46:54.464908 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:46:54.464908 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:46:54.464908 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:46:54.464908 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:46:54.464908 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 15:46:54.465871 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 15:46:54.465871 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 15:46:54.465871 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Aug 30 15:46:54.470056 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 360/713] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 30 15:46:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 15:46:54.471005 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b43da5-2abd-4e65-a1ac-b7baf2f81134 None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:54.544108 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:54.578570 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a68975-1f70-4cec-921f-4e5b804a0e1a None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:54.583399 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b43da5-2abd-4e65-a1ac-b7baf2f81134 None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:54.619104 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959efc31-4f86-4dd1-911c-870731a72218 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:54.620196 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959efc31-4f86-4dd1-911c-870731a72218 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:54.621171 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959efc31-4f86-4dd1-911c-870731a72218 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:54.621714 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959efc31-4f86-4dd1-911c-870731a72218 None admin] PATH_INFO: `/v3/services` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:54.622779 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959efc31-4f86-4dd1-911c-870731a72218 None admin] RBAC: Adding query filter params (name=network) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:46:54.623240 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959efc31-4f86-4dd1-911c-870731a72218 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:54.623354 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb07d6d-92e5-459c-a32b-5ae12911eab4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:54.624075 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb07d6d-92e5-459c-a32b-5ae12911eab4 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:54.624449 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb07d6d-92e5-459c-a32b-5ae12911eab4 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:54.624814 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb07d6d-92e5-459c-a32b-5ae12911eab4 None admin] PATH_INFO: `/v3/domains` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:54.625459 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959efc31-4f86-4dd1-911c-870731a72218 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:54.625703 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb07d6d-92e5-459c-a32b-5ae12911eab4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81512) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:46:54.626101 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb07d6d-92e5-459c-a32b-5ae12911eab4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:54.627579 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb07d6d-92e5-459c-a32b-5ae12911eab4 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:54.632335 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 354/714] 10.222.0.20 () {56 vars in 1112 bytes} [Tue Aug 30 15:46:54 2022] GET /identity/v3/services?name=network => generated 124 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:46:54.646900 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 361/715] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Aug 30 15:46:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:46:54.647485 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959efc31-4f86-4dd1-911c-870731a72218 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:54.655403 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb07d6d-92e5-459c-a32b-5ae12911eab4 None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:54.749976 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:54.768698 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb07d6d-92e5-459c-a32b-5ae12911eab4 None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:54.800752 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:54.809691 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959efc31-4f86-4dd1-911c-870731a72218 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:54.814710 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b615bf0-7234-4afa-a45a-1c59658a3526 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:54.815592 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b615bf0-7234-4afa-a45a-1c59658a3526 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:54.815866 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b615bf0-7234-4afa-a45a-1c59658a3526 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:54.816178 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b615bf0-7234-4afa-a45a-1c59658a3526 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:54.823897 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b615bf0-7234-4afa-a45a-1c59658a3526 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:54.825396 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b615bf0-7234-4afa-a45a-1c59658a3526 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:54.839402 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application [None req-0b615bf0-7234-4afa-a45a-1c59658a3526 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 15:46:54.839402 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:46:54.839402 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:46:54.839402 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:46:54.839402 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:46:54.839402 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:46:54.839402 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:46:54.839402 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:46:54.839402 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:46:54.839402 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:46:54.839402 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:46:54.839402 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:46:54.839402 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 15:46:54.839402 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 15:46:54.839402 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 15:46:54.840325 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 15:46:54.840325 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:46:54.840325 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:46:54.840325 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:46:54.840325 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:46:54.840325 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:46:54.840325 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:46:54.840325 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:46:54.840325 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application with Lock( Aug 30 15:46:54.840325 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:46:54.840325 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:46:54.840325 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:46:54.840325 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:46:54.840325 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:46:54.840325 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:46:54.840325 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:46:54.840325 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:46:54.841914 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 15:46:54.841914 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 15:46:54.841914 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 15:46:54.841914 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 15:46:54.841914 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 15:46:54.841914 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 15:46:54.841914 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 15:46:54.841914 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Aug 30 15:46:54.841914 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 362/716] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Aug 30 15:46:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 15:46:54.842544 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50293af5-05f0-4be2-989d-45057e5b71a0 None None] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:54.842544 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50293af5-05f0-4be2-989d-45057e5b71a0 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:54.842679 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50293af5-05f0-4be2-989d-45057e5b71a0 None None] PATH_INFO: `/` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:54.844019 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 363/717] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 30 15:46:54 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 15:46:54.848714 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50293af5-05f0-4be2-989d-45057e5b71a0 None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:54.852819 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863fcc39-0c3e-497f-a796-17a8f2bd9fad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:54.853577 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863fcc39-0c3e-497f-a796-17a8f2bd9fad None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:54.853857 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863fcc39-0c3e-497f-a796-17a8f2bd9fad None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:54.854130 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863fcc39-0c3e-497f-a796-17a8f2bd9fad None admin] PATH_INFO: `/v3/services` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:54.854775 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863fcc39-0c3e-497f-a796-17a8f2bd9fad None admin] RBAC: Adding query filter params (type=network) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:46:54.855075 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863fcc39-0c3e-497f-a796-17a8f2bd9fad None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:54.856456 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863fcc39-0c3e-497f-a796-17a8f2bd9fad None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:54.863726 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 355/718] 10.222.0.20 () {56 vars in 1112 bytes} [Tue Aug 30 15:46:54 2022] GET /identity/v3/services?type=network => generated 124 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 15:46:54.864957 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877aa19a-f9ca-44e3-b8c5-2dd0623fe436 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:54.865283 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877aa19a-f9ca-44e3-b8c5-2dd0623fe436 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:54.865565 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877aa19a-f9ca-44e3-b8c5-2dd0623fe436 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:54.893357 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-877aa19a-f9ca-44e3-b8c5-2dd0623fe436 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:46:54.894587 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-877aa19a-f9ca-44e3-b8c5-2dd0623fe436 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:46:54.974228 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 356/719] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 30 15:46:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 15:46:55.010651 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50293af5-05f0-4be2-989d-45057e5b71a0 None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:55.065867 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc435b8-a62d-4148-a61a-d0e136a5c567 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:55.066736 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc435b8-a62d-4148-a61a-d0e136a5c567 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:55.067152 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc435b8-a62d-4148-a61a-d0e136a5c567 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:55.067557 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc435b8-a62d-4148-a61a-d0e136a5c567 None admin] PATH_INFO: `/v3/projects` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:55.068411 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc435b8-a62d-4148-a61a-d0e136a5c567 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81512) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:46:55.068855 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc435b8-a62d-4148-a61a-d0e136a5c567 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:55.070528 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc435b8-a62d-4148-a61a-d0e136a5c567 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:55.085490 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 364/720] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Aug 30 15:46:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:46:55.091367 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877aa19a-f9ca-44e3-b8c5-2dd0623fe436 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:55.155438 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:55.161422 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c31c11-200c-4315-a6b2-5acbcc8d1a1b None None] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:55.161737 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c31c11-200c-4315-a6b2-5acbcc8d1a1b None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:55.161962 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c31c11-200c-4315-a6b2-5acbcc8d1a1b None None] PATH_INFO: `/` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:55.163434 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 365/721] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 30 15:46:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 15:46:55.168739 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c7745a-7e6c-46f8-b034-9235ee765408 None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:55.169015 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c7745a-7e6c-46f8-b034-9235ee765408 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:55.169323 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c7745a-7e6c-46f8-b034-9235ee765408 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:55.202383 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-58c7745a-7e6c-46f8-b034-9235ee765408 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:46:55.203508 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-58c7745a-7e6c-46f8-b034-9235ee765408 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:46:55.211745 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877aa19a-f9ca-44e3-b8c5-2dd0623fe436 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:55.251205 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6a6554-72ce-4467-8253-39b7d2af08e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:55.252907 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6a6554-72ce-4467-8253-39b7d2af08e1 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:55.253410 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6a6554-72ce-4467-8253-39b7d2af08e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:55.254721 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6a6554-72ce-4467-8253-39b7d2af08e1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:55.255698 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6a6554-72ce-4467-8253-39b7d2af08e1 None admin] RBAC: Adding query filter params (role.id=f265386950a641ac8413b53eda4ae34d, scope.project.id=854fe71b49224dff9a4b1766a20ab266, user.id=7a1e756ae9be4a7bac1e81acf07e8f3b) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:46:55.256426 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6a6554-72ce-4467-8253-39b7d2af08e1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:55.259911 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6a6554-72ce-4467-8253-39b7d2af08e1 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:55.270801 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 357/722] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Aug 30 15:46:55 2022] GET /identity/v3/role_assignments?user.id=7a1e756ae9be4a7bac1e81acf07e8f3b&scope.project.id=854fe71b49224dff9a4b1766a20ab266&role.id=f265386950a641ac8413b53eda4ae34d => generated 259 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:46:55.289713 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:55.323124 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 366/723] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 30 15:46:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 15:46:55.356974 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6a6554-72ce-4467-8253-39b7d2af08e1 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:55.467909 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6a6554-72ce-4467-8253-39b7d2af08e1 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:55.487167 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:55.506130 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4912368-7cb1-46be-81a8-fa57ee8fd148 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:55.507129 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4912368-7cb1-46be-81a8-fa57ee8fd148 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:55.507656 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4912368-7cb1-46be-81a8-fa57ee8fd148 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:55.508155 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4912368-7cb1-46be-81a8-fa57ee8fd148 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:55.518931 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4912368-7cb1-46be-81a8-fa57ee8fd148 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 15:46:55.524230 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-c4912368-7cb1-46be-81a8-fa57ee8fd148 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 15:46:55.524230 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:46:55.524230 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:46:55.524230 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:46:55.524230 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:46:55.524230 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:46:55.524230 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:46:55.524230 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:46:55.524230 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:46:55.524230 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:46:55.524230 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:46:55.524230 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:46:55.524230 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 15:46:55.524230 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 15:46:55.524230 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 15:46:55.524230 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 15:46:55.525173 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 15:46:55.525173 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 15:46:55.525173 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 15:46:55.525173 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 15:46:55.525173 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:46:55.525173 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:46:55.525173 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:46:55.525173 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:46:55.525173 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:46:55.525173 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:46:55.525173 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:46:55.525173 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Aug 30 15:46:55.525173 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:46:55.525173 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:46:55.525173 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:46:55.525173 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:46:55.526058 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:46:55.526058 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:46:55.526058 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:46:55.526058 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:46:55.526058 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 15:46:55.526058 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 15:46:55.526058 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 15:46:55.526058 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 15:46:55.526058 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 15:46:55.526058 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 15:46:55.526058 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 15:46:55.526058 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Aug 30 15:46:55.528728 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 358/724] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Aug 30 15:46:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 15:46:55.534339 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c7745a-7e6c-46f8-b034-9235ee765408 None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:55.659427 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c7745a-7e6c-46f8-b034-9235ee765408 None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:55.717695 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c54d85-d7f2-423b-8699-e6cf3dcfc3e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:55.718315 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c54d85-d7f2-423b-8699-e6cf3dcfc3e8 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:55.718594 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c54d85-d7f2-423b-8699-e6cf3dcfc3e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:55.718863 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c54d85-d7f2-423b-8699-e6cf3dcfc3e8 None admin] PATH_INFO: `/v3/roles` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:55.719536 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c54d85-d7f2-423b-8699-e6cf3dcfc3e8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81512) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:46:55.719835 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c54d85-d7f2-423b-8699-e6cf3dcfc3e8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:55.721236 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c54d85-d7f2-423b-8699-e6cf3dcfc3e8 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:55.738660 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 367/725] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Aug 30 15:46:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 15:46:55.746751 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4912368-7cb1-46be-81a8-fa57ee8fd148 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:55.784147 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c54d85-d7f2-423b-8699-e6cf3dcfc3e8 None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:55.868424 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:55.902901 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4912368-7cb1-46be-81a8-fa57ee8fd148 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:55.944681 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32af127d-97a5-4abb-bc4e-5712521e86ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:55.945912 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32af127d-97a5-4abb-bc4e-5712521e86ff None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:55.947091 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32af127d-97a5-4abb-bc4e-5712521e86ff None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:55.949072 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32af127d-97a5-4abb-bc4e-5712521e86ff None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:55.967045 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32af127d-97a5-4abb-bc4e-5712521e86ff None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:55.971057 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32af127d-97a5-4abb-bc4e-5712521e86ff None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:55.988181 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c54d85-d7f2-423b-8699-e6cf3dcfc3e8 None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:55.992814 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-32af127d-97a5-4abb-bc4e-5712521e86ff None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 15:46:55.992814 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:46:55.992814 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:46:55.992814 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:46:55.992814 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:46:55.992814 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:46:55.992814 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:46:55.992814 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:46:55.992814 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:46:55.992814 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:46:55.992814 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:46:55.992814 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:46:55.992814 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 15:46:55.992814 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 15:46:55.992814 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 15:46:55.992814 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 15:46:55.994127 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:46:55.994127 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:46:55.994127 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:46:55.994127 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:46:55.994127 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:46:55.994127 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:46:55.994127 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:46:55.994127 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Aug 30 15:46:55.994127 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:46:55.994127 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:46:55.994127 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:46:55.994127 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:46:55.994127 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:46:55.994127 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:46:55.994127 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:46:55.994127 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:46:55.994127 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 15:46:55.995022 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 15:46:55.995022 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 15:46:55.995022 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Aug 30 15:46:55.998831 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 359/726] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 30 15:46:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 15:46:56.002977 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32af127d-97a5-4abb-bc4e-5712521e86ff None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:56.052997 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901222b0-4979-473d-b597-d8526da01bfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:56.054366 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901222b0-4979-473d-b597-d8526da01bfa None admin] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:56.056188 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901222b0-4979-473d-b597-d8526da01bfa None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:56.056677 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901222b0-4979-473d-b597-d8526da01bfa None admin] PATH_INFO: `/v3/services` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:56.058925 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901222b0-4979-473d-b597-d8526da01bfa None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:56.060671 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901222b0-4979-473d-b597-d8526da01bfa None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:56.067966 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:56.090526 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 368/727] 10.222.0.20 () {60 vars in 1141 bytes} [Tue Aug 30 15:46:55 2022] POST /identity/v3/services => generated 251 bytes in 307 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 15:46:56.159494 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32af127d-97a5-4abb-bc4e-5712521e86ff None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:56.214669 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b491d9fb-056b-491b-bad3-4cc4991b2aa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:56.215671 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b491d9fb-056b-491b-bad3-4cc4991b2aa1 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:56.216137 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b491d9fb-056b-491b-bad3-4cc4991b2aa1 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:56.216624 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b491d9fb-056b-491b-bad3-4cc4991b2aa1 None admin] PATH_INFO: `/v3/domains` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:56.217545 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b491d9fb-056b-491b-bad3-4cc4991b2aa1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:46:56.218087 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b491d9fb-056b-491b-bad3-4cc4991b2aa1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:56.219659 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b491d9fb-056b-491b-bad3-4cc4991b2aa1 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:56.236906 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 360/728] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Aug 30 15:46:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:46:56.242344 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901222b0-4979-473d-b597-d8526da01bfa None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:56.378714 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901222b0-4979-473d-b597-d8526da01bfa None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:56.386452 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:56.415532 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1b12d0-176e-4051-8579-55628edd7fbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:56.415976 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1b12d0-176e-4051-8579-55628edd7fbe None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:56.416198 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1b12d0-176e-4051-8579-55628edd7fbe None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:56.416555 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1b12d0-176e-4051-8579-55628edd7fbe None admin] PATH_INFO: `/v3/users/swift` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:56.425426 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1b12d0-176e-4051-8579-55628edd7fbe None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:56.426882 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1b12d0-176e-4051-8579-55628edd7fbe None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:56.438162 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application [None req-0c1b12d0-176e-4051-8579-55628edd7fbe None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 30 15:46:56.438162 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:46:56.438162 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:46:56.438162 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:46:56.438162 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:46:56.438162 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:46:56.438162 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:46:56.438162 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:46:56.438162 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:46:56.438162 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:46:56.438162 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:46:56.438162 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:46:56.438162 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 15:46:56.438162 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 15:46:56.438162 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 15:46:56.438162 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 15:46:56.439141 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:46:56.439141 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:46:56.439141 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 15:46:56.439141 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 15:46:56.439141 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 15:46:56.439141 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 15:46:56.439141 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:46:56.439141 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:46:56.439141 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:46:56.439141 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:46:56.439141 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:46:56.439141 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application with Lock( Aug 30 15:46:56.439141 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:46:56.439141 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:46:56.439141 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:46:56.439141 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:46:56.439141 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:46:56.439141 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:46:56.440738 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:46:56.440738 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:46:56.440738 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 15:46:56.440738 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 15:46:56.440738 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 15:46:56.440738 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 15:46:56.440738 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 15:46:56.440738 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 15:46:56.440738 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 30 15:46:56.440738 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Aug 30 15:46:56.440738 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 369/729] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Aug 30 15:46:56 2022] GET /identity/v3/users/swift => generated 83 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 15:46:56.444891 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b491d9fb-056b-491b-bad3-4cc4991b2aa1 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:56.537339 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:56.554387 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b491d9fb-056b-491b-bad3-4cc4991b2aa1 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:56.588916 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eaf1755-c6e8-41d5-822e-dee25f1a0677 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:56.589553 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eaf1755-c6e8-41d5-822e-dee25f1a0677 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:56.589826 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eaf1755-c6e8-41d5-822e-dee25f1a0677 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:56.590088 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eaf1755-c6e8-41d5-822e-dee25f1a0677 None admin] PATH_INFO: `/v3/users` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:56.590821 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eaf1755-c6e8-41d5-822e-dee25f1a0677 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:46:56.591119 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eaf1755-c6e8-41d5-822e-dee25f1a0677 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:56.592583 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eaf1755-c6e8-41d5-822e-dee25f1a0677 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:56.611869 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 361/730] 10.222.0.20 () {56 vars in 1135 bytes} [Tue Aug 30 15:46:56 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 15:46:56.617194 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1b12d0-176e-4051-8579-55628edd7fbe None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:56.735202 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1b12d0-176e-4051-8579-55628edd7fbe None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:56.771111 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac8d2e5-451e-4fb6-bc9e-d82e931fc2c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:56.772059 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac8d2e5-451e-4fb6-bc9e-d82e931fc2c9 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:56.772519 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac8d2e5-451e-4fb6-bc9e-d82e931fc2c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:56.772960 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac8d2e5-451e-4fb6-bc9e-d82e931fc2c9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:56.802200 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:56.844347 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac8d2e5-451e-4fb6-bc9e-d82e931fc2c9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:56.846146 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac8d2e5-451e-4fb6-bc9e-d82e931fc2c9 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:56.946971 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 370/731] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Aug 30 15:46:56 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 30 15:46:56.954349 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eaf1755-c6e8-41d5-822e-dee25f1a0677 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:57.010881 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1342ddb3-c203-4cbb-8bd2-54f1ec13cb5e None None] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:57.011782 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1342ddb3-c203-4cbb-8bd2-54f1ec13cb5e None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:57.012340 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1342ddb3-c203-4cbb-8bd2-54f1ec13cb5e None None] PATH_INFO: `/` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:57.014822 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 371/732] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 30 15:46:57 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 15:46:57.020785 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a65e7d-bbd7-4bd9-abd6-3be089177a09 None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:57.021072 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a65e7d-bbd7-4bd9-abd6-3be089177a09 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:57.021476 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a65e7d-bbd7-4bd9-abd6-3be089177a09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:57.049028 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-05a65e7d-bbd7-4bd9-abd6-3be089177a09 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:46:57.050266 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-05a65e7d-bbd7-4bd9-abd6-3be089177a09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:46:57.118894 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eaf1755-c6e8-41d5-822e-dee25f1a0677 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:57.153656 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:57.167045 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 372/733] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 30 15:46:57 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 15:46:57.175323 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b915c57-d913-4fac-a502-a5ad0aa6fcda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:57.175992 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b915c57-d913-4fac-a502-a5ad0aa6fcda None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:57.176411 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b915c57-d913-4fac-a502-a5ad0aa6fcda None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:57.176874 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b915c57-d913-4fac-a502-a5ad0aa6fcda None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:57.197129 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b915c57-d913-4fac-a502-a5ad0aa6fcda None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:57.198722 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b915c57-d913-4fac-a502-a5ad0aa6fcda None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:57.213537 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:57.220656 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-0b915c57-d913-4fac-a502-a5ad0aa6fcda None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 15:46:57.220656 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:46:57.220656 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:46:57.220656 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:46:57.220656 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:46:57.220656 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:46:57.220656 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:46:57.220656 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:46:57.220656 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:46:57.220656 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:46:57.220656 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:46:57.220656 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:46:57.220656 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 15:46:57.220656 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 15:46:57.220656 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 15:46:57.220656 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 15:46:57.221645 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:46:57.221645 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:46:57.221645 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:46:57.221645 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:46:57.221645 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:46:57.221645 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:46:57.221645 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:46:57.221645 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Aug 30 15:46:57.221645 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:46:57.221645 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:46:57.221645 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:46:57.221645 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:46:57.221645 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:46:57.221645 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:46:57.221645 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:46:57.221645 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:46:57.221645 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 15:46:57.222567 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 15:46:57.222567 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 15:46:57.222567 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Aug 30 15:46:57.222567 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 362/734] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 30 15:46:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 15:46:57.227847 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a65e7d-bbd7-4bd9-abd6-3be089177a09 None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:57.238963 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4249b347-7c5b-41a6-afe9-9e1f9f56f8f6 None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:57.239866 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4249b347-7c5b-41a6-afe9-9e1f9f56f8f6 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:57.240311 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4249b347-7c5b-41a6-afe9-9e1f9f56f8f6 None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:57.242708 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 363/735] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 30 15:46:57 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 15:46:57.248225 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90da8c96-042a-4301-98c6-ffb6d3d23aa1 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:57.248751 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90da8c96-042a-4301-98c6-ffb6d3d23aa1 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:57.249264 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90da8c96-042a-4301-98c6-ffb6d3d23aa1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:57.268617 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-90da8c96-042a-4301-98c6-ffb6d3d23aa1 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:46:57.270325 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-90da8c96-042a-4301-98c6-ffb6d3d23aa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:46:57.354598 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 364/736] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 30 15:46:57 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 15:46:57.422457 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a65e7d-bbd7-4bd9-abd6-3be089177a09 None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:57.472270 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec94bc7-1826-47e4-8796-2f94ecf91b5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:57.472887 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec94bc7-1826-47e4-8796-2f94ecf91b5b None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:57.473159 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec94bc7-1826-47e4-8796-2f94ecf91b5b None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:57.473457 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec94bc7-1826-47e4-8796-2f94ecf91b5b None admin] PATH_INFO: `/v3/domains` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:57.474111 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec94bc7-1826-47e4-8796-2f94ecf91b5b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81512) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:46:57.474413 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec94bc7-1826-47e4-8796-2f94ecf91b5b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:57.475796 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec94bc7-1826-47e4-8796-2f94ecf91b5b None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:57.477463 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8607271a-971e-4d59-b4f2-ad330e2adea3 None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:57.477754 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8607271a-971e-4d59-b4f2-ad330e2adea3 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:57.478032 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8607271a-971e-4d59-b4f2-ad330e2adea3 None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:57.481635 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 365/737] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 30 15:46:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 15:46:57.485004 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b3ceac-61be-4d56-8241-03b407eeb764 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:57.485327 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b3ceac-61be-4d56-8241-03b407eeb764 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:57.485610 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b3ceac-61be-4d56-8241-03b407eeb764 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:57.494451 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 373/738] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Aug 30 15:46:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:46:57.500504 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec94bc7-1826-47e4-8796-2f94ecf91b5b None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:57.514252 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-06b3ceac-61be-4d56-8241-03b407eeb764 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:46:57.515454 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-06b3ceac-61be-4d56-8241-03b407eeb764 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:46:57.577724 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:57.629917 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 366/739] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 30 15:46:57 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 15:46:57.630289 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b3ceac-61be-4d56-8241-03b407eeb764 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:57.644526 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec94bc7-1826-47e4-8796-2f94ecf91b5b None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:57.682542 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fae8fd-42ea-4171-a13f-e2b7e9e35a5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:57.683171 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fae8fd-42ea-4171-a13f-e2b7e9e35a5e None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:57.683448 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fae8fd-42ea-4171-a13f-e2b7e9e35a5e None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:57.683717 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fae8fd-42ea-4171-a13f-e2b7e9e35a5e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:57.690896 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fae8fd-42ea-4171-a13f-e2b7e9e35a5e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:57.692364 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fae8fd-42ea-4171-a13f-e2b7e9e35a5e None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:57.705102 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application [None req-f7fae8fd-42ea-4171-a13f-e2b7e9e35a5e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 15:46:57.705102 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:46:57.705102 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:46:57.705102 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:46:57.705102 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:46:57.705102 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:46:57.705102 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:46:57.705102 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:46:57.705102 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:46:57.705102 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:46:57.705102 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:46:57.705102 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:46:57.705102 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 15:46:57.705102 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 15:46:57.705102 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 15:46:57.706122 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 15:46:57.706122 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:46:57.706122 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:46:57.706122 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:46:57.706122 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:46:57.706122 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:46:57.706122 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:46:57.706122 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:46:57.706122 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application with Lock( Aug 30 15:46:57.706122 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:46:57.706122 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:46:57.706122 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:46:57.706122 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:46:57.706122 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:46:57.706122 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:46:57.706122 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:46:57.706122 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:46:57.707004 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 15:46:57.707004 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 15:46:57.707004 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 15:46:57.707004 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 15:46:57.707004 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 15:46:57.707004 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 15:46:57.707004 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 15:46:57.707004 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Aug 30 15:46:57.707004 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 374/740] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Aug 30 15:46:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 15:46:57.709677 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:57.713753 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fae8fd-42ea-4171-a13f-e2b7e9e35a5e None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:57.764138 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b3ceac-61be-4d56-8241-03b407eeb764 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:57.805241 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5970331f-13f7-4e8e-99d3-8ba0dc5f5d46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:57.805241 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5970331f-13f7-4e8e-99d3-8ba0dc5f5d46 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:57.805241 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5970331f-13f7-4e8e-99d3-8ba0dc5f5d46 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:57.805241 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5970331f-13f7-4e8e-99d3-8ba0dc5f5d46 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:57.816150 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5970331f-13f7-4e8e-99d3-8ba0dc5f5d46 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 15:46:57.822148 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-5970331f-13f7-4e8e-99d3-8ba0dc5f5d46 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 15:46:57.822148 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:46:57.822148 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:46:57.822148 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:46:57.822148 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:46:57.822148 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:46:57.822148 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:46:57.822148 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:46:57.822148 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:46:57.822148 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:46:57.822148 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:46:57.822148 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:46:57.822148 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 15:46:57.822148 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 15:46:57.822148 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 15:46:57.822148 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 15:46:57.823100 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 15:46:57.823100 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 15:46:57.823100 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 15:46:57.823100 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 15:46:57.823100 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:46:57.823100 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:46:57.823100 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:46:57.823100 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:46:57.823100 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:46:57.823100 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:46:57.823100 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:46:57.823100 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Aug 30 15:46:57.823100 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:46:57.823100 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:46:57.823100 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:46:57.823100 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:46:57.824129 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:46:57.824129 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:46:57.824129 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:46:57.824129 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:46:57.824129 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 15:46:57.824129 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 15:46:57.824129 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 15:46:57.824129 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 15:46:57.824129 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 15:46:57.824129 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 15:46:57.824129 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 15:46:57.824129 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Aug 30 15:46:57.825298 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 367/741] 10.222.0.20 () {56 vars in 1102 bytes} [Tue Aug 30 15:46:57 2022] GET /identity/v3/roles/service => generated 85 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 15:46:57.826127 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5970331f-13f7-4e8e-99d3-8ba0dc5f5d46 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:57.843327 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fae8fd-42ea-4171-a13f-e2b7e9e35a5e None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:57.858240 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:57.888306 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb4b645-4b22-4b5c-a522-aff1a693eab3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:57.889469 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb4b645-4b22-4b5c-a522-aff1a693eab3 None admin] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:57.890374 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb4b645-4b22-4b5c-a522-aff1a693eab3 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:57.890805 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb4b645-4b22-4b5c-a522-aff1a693eab3 None admin] PATH_INFO: `/v3/services` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:57.891842 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb4b645-4b22-4b5c-a522-aff1a693eab3 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:57.894017 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb4b645-4b22-4b5c-a522-aff1a693eab3 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:57.915148 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 375/742] 10.222.0.20 () {60 vars in 1141 bytes} [Tue Aug 30 15:46:57 2022] POST /identity/v3/services => generated 240 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 15:46:57.917396 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb4b645-4b22-4b5c-a522-aff1a693eab3 None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:57.953305 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5970331f-13f7-4e8e-99d3-8ba0dc5f5d46 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:57.973467 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:58.004378 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fa2c9a-901d-4867-a3d9-16376b423ff6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:58.005024 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fa2c9a-901d-4867-a3d9-16376b423ff6 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:58.005334 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fa2c9a-901d-4867-a3d9-16376b423ff6 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:58.005602 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fa2c9a-901d-4867-a3d9-16376b423ff6 None admin] PATH_INFO: `/v3/projects` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:58.006324 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fa2c9a-901d-4867-a3d9-16376b423ff6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:46:58.006628 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fa2c9a-901d-4867-a3d9-16376b423ff6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:58.008404 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fa2c9a-901d-4867-a3d9-16376b423ff6 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:58.022715 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 368/743] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Aug 30 15:46:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:46:58.023615 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fa2c9a-901d-4867-a3d9-16376b423ff6 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:58.056732 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb4b645-4b22-4b5c-a522-aff1a693eab3 None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:58.094787 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261288f2-5f95-4f18-a0f2-82b197173ec5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:58.095738 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261288f2-5f95-4f18-a0f2-82b197173ec5 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:58.096205 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261288f2-5f95-4f18-a0f2-82b197173ec5 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:58.096679 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261288f2-5f95-4f18-a0f2-82b197173ec5 None admin] PATH_INFO: `/v3/roles` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:58.097792 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261288f2-5f95-4f18-a0f2-82b197173ec5 None admin] RBAC: Adding query filter params (name=service) {{(pid=81512) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:46:58.098313 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261288f2-5f95-4f18-a0f2-82b197173ec5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:58.100000 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261288f2-5f95-4f18-a0f2-82b197173ec5 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:58.113395 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 376/744] 10.222.0.20 () {56 vars in 1103 bytes} [Tue Aug 30 15:46:57 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:46:58.114734 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261288f2-5f95-4f18-a0f2-82b197173ec5 None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:58.148563 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fa2c9a-901d-4867-a3d9-16376b423ff6 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:58.158431 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:58.188392 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360ed8c9-824a-47f4-9b2e-b82dcbf56c89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:58.189329 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360ed8c9-824a-47f4-9b2e-b82dcbf56c89 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:58.189623 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360ed8c9-824a-47f4-9b2e-b82dcbf56c89 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:58.189900 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360ed8c9-824a-47f4-9b2e-b82dcbf56c89 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:58.204346 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360ed8c9-824a-47f4-9b2e-b82dcbf56c89 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 15:46:58.211148 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-360ed8c9-824a-47f4-9b2e-b82dcbf56c89 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 15:46:58.211148 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:46:58.211148 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:46:58.211148 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:46:58.211148 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:46:58.211148 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:46:58.211148 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:46:58.211148 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:46:58.211148 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:46:58.211148 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:46:58.211148 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:46:58.211148 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:46:58.211148 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 15:46:58.211148 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 15:46:58.211148 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 15:46:58.211148 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 15:46:58.212124 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 15:46:58.212124 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 15:46:58.212124 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 15:46:58.212124 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 15:46:58.212124 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:46:58.212124 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:46:58.212124 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:46:58.212124 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:46:58.212124 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:46:58.212124 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:46:58.212124 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:46:58.212124 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Aug 30 15:46:58.212124 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:46:58.212124 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:46:58.212124 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:46:58.212124 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:46:58.212957 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:46:58.212957 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:46:58.212957 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:46:58.212957 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:46:58.212957 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 15:46:58.212957 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 15:46:58.212957 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 15:46:58.212957 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 15:46:58.212957 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 15:46:58.212957 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 15:46:58.212957 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 15:46:58.212957 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Aug 30 15:46:58.215520 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 369/745] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Aug 30 15:46:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 15:46:58.216324 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360ed8c9-824a-47f4-9b2e-b82dcbf56c89 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:58.243104 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261288f2-5f95-4f18-a0f2-82b197173ec5 None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:58.278295 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40948d14-d5fa-4ce1-98c3-7fd2d3031d1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:58.278907 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40948d14-d5fa-4ce1-98c3-7fd2d3031d1c None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:58.279186 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40948d14-d5fa-4ce1-98c3-7fd2d3031d1c None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:58.279486 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40948d14-d5fa-4ce1-98c3-7fd2d3031d1c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:58.279963 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:58.280284 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40948d14-d5fa-4ce1-98c3-7fd2d3031d1c None admin] RBAC: Adding query filter params (role.id=f265386950a641ac8413b53eda4ae34d, scope.project.id=854fe71b49224dff9a4b1766a20ab266, user.id=8fdafdc39a594bb4b0aeaf2fbc84c2fb) {{(pid=81512) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:46:58.280642 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40948d14-d5fa-4ce1-98c3-7fd2d3031d1c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:58.282099 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40948d14-d5fa-4ce1-98c3-7fd2d3031d1c None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:58.287968 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 377/746] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Aug 30 15:46:58 2022] GET /identity/v3/role_assignments?user.id=8fdafdc39a594bb4b0aeaf2fbc84c2fb&scope.project.id=854fe71b49224dff9a4b1766a20ab266&role.id=f265386950a641ac8413b53eda4ae34d => generated 259 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:46:58.288925 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40948d14-d5fa-4ce1-98c3-7fd2d3031d1c None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:58.397619 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:58.401839 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40948d14-d5fa-4ce1-98c3-7fd2d3031d1c None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:58.423487 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360ed8c9-824a-47f4-9b2e-b82dcbf56c89 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:58.442488 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0563d2af-23be-4f15-82a3-34a766e91949 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:58.443279 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0563d2af-23be-4f15-82a3-34a766e91949 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:58.443658 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0563d2af-23be-4f15-82a3-34a766e91949 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:58.444033 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0563d2af-23be-4f15-82a3-34a766e91949 None admin] PATH_INFO: `/v3/roles` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:58.444894 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0563d2af-23be-4f15-82a3-34a766e91949 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81512) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:46:58.445351 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0563d2af-23be-4f15-82a3-34a766e91949 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:58.447524 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0563d2af-23be-4f15-82a3-34a766e91949 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:58.458090 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 378/747] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Aug 30 15:46:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:46:58.463805 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0563d2af-23be-4f15-82a3-34a766e91949 None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:58.492843 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a9ea8b-ea82-4ac9-8afb-fbc6141dd562 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:58.493595 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a9ea8b-ea82-4ac9-8afb-fbc6141dd562 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:58.493880 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a9ea8b-ea82-4ac9-8afb-fbc6141dd562 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:58.494199 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a9ea8b-ea82-4ac9-8afb-fbc6141dd562 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:58.509675 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a9ea8b-ea82-4ac9-8afb-fbc6141dd562 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:58.511311 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a9ea8b-ea82-4ac9-8afb-fbc6141dd562 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:58.525979 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-e8a9ea8b-ea82-4ac9-8afb-fbc6141dd562 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 15:46:58.525979 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:46:58.525979 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:46:58.525979 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:46:58.525979 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:46:58.525979 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:46:58.525979 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:46:58.525979 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:46:58.525979 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:46:58.525979 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:46:58.525979 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:46:58.525979 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:46:58.525979 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 15:46:58.525979 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 15:46:58.525979 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 15:46:58.525979 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 15:46:58.526956 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:46:58.526956 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:46:58.526956 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:46:58.526956 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:46:58.526956 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:46:58.526956 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:46:58.526956 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:46:58.526956 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Aug 30 15:46:58.526956 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:46:58.526956 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:46:58.526956 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:46:58.526956 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:46:58.526956 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:46:58.526956 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:46:58.526956 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:46:58.526956 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:46:58.526956 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 15:46:58.527848 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 15:46:58.527848 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 15:46:58.527848 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Aug 30 15:46:58.535806 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 370/748] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 30 15:46:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 320 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 15:46:58.536746 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a9ea8b-ea82-4ac9-8afb-fbc6141dd562 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:58.574571 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0563d2af-23be-4f15-82a3-34a766e91949 None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:58.619818 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b505ca8-5783-42b6-bcfb-6ef9a14c1931 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:58.620859 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b505ca8-5783-42b6-bcfb-6ef9a14c1931 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:58.621138 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b505ca8-5783-42b6-bcfb-6ef9a14c1931 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:58.621437 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b505ca8-5783-42b6-bcfb-6ef9a14c1931 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:58.643344 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b505ca8-5783-42b6-bcfb-6ef9a14c1931 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:58.643344 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b505ca8-5783-42b6-bcfb-6ef9a14c1931 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:58.673299 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application [None req-1b505ca8-5783-42b6-bcfb-6ef9a14c1931 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 15:46:58.673299 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:46:58.673299 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:46:58.673299 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:46:58.673299 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:46:58.673299 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:46:58.673299 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:46:58.673299 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:46:58.673299 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:46:58.673299 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:46:58.673299 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:46:58.673299 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:46:58.673299 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 15:46:58.673299 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 15:46:58.673299 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 15:46:58.673299 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 15:46:58.674276 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:46:58.674276 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:46:58.674276 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:46:58.674276 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:46:58.674276 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:46:58.674276 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:46:58.674276 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:46:58.674276 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application with Lock( Aug 30 15:46:58.674276 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:46:58.674276 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:46:58.674276 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:46:58.674276 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:46:58.674276 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:46:58.674276 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:46:58.674276 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:46:58.674276 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:46:58.674276 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 15:46:58.675158 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 15:46:58.675158 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 15:46:58.675158 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Aug 30 15:46:58.678811 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 379/749] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 30 15:46:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 15:46:58.685198 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b505ca8-5783-42b6-bcfb-6ef9a14c1931 None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:58.704140 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:58.707729 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:58.731571 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a9ea8b-ea82-4ac9-8afb-fbc6141dd562 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:58.775680 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230d3d1a-f6c9-48a1-aabd-975f7d9e339c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:58.776303 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230d3d1a-f6c9-48a1-aabd-975f7d9e339c None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:58.776743 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230d3d1a-f6c9-48a1-aabd-975f7d9e339c None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:58.777019 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230d3d1a-f6c9-48a1-aabd-975f7d9e339c None admin] PATH_INFO: `/v3/domains` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:58.778082 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230d3d1a-f6c9-48a1-aabd-975f7d9e339c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:46:58.778508 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230d3d1a-f6c9-48a1-aabd-975f7d9e339c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:58.780344 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230d3d1a-f6c9-48a1-aabd-975f7d9e339c None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:58.794222 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 371/750] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Aug 30 15:46:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:46:58.795587 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43dabc5-b19d-47f9-8413-8e45e5156cab None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:58.795870 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43dabc5-b19d-47f9-8413-8e45e5156cab None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:58.796140 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43dabc5-b19d-47f9-8413-8e45e5156cab None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:58.796397 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b505ca8-5783-42b6-bcfb-6ef9a14c1931 None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:58.797612 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 372/751] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 30 15:46:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 15:46:58.799690 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43dabc5-b19d-47f9-8413-8e45e5156cab None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:58.836341 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba00755-1acb-4f9e-8079-5481288dfa51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:58.837097 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba00755-1acb-4f9e-8079-5481288dfa51 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:58.837424 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba00755-1acb-4f9e-8079-5481288dfa51 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:58.837710 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba00755-1acb-4f9e-8079-5481288dfa51 None admin] PATH_INFO: `/v3/domains` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:58.838339 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba00755-1acb-4f9e-8079-5481288dfa51 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81512) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:46:58.838639 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba00755-1acb-4f9e-8079-5481288dfa51 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:58.840016 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba00755-1acb-4f9e-8079-5481288dfa51 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:58.853921 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 380/752] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Aug 30 15:46:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:46:58.855266 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9153c8-5d88-44d3-bd09-43c8f6d1f778 None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:58.855550 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9153c8-5d88-44d3-bd09-43c8f6d1f778 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:58.855823 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9153c8-5d88-44d3-bd09-43c8f6d1f778 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:58.877561 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-1f9153c8-5d88-44d3-bd09-43c8f6d1f778 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:46:58.878955 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-1f9153c8-5d88-44d3-bd09-43c8f6d1f778 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:46:58.910037 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43dabc5-b19d-47f9-8413-8e45e5156cab None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:58.945450 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6cfc33-1645-4020-a093-9e16ca0e55a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:58.946010 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6cfc33-1645-4020-a093-9e16ca0e55a2 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:58.946291 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6cfc33-1645-4020-a093-9e16ca0e55a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:58.946564 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6cfc33-1645-4020-a093-9e16ca0e55a2 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:58.947721 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:58.953585 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 381/753] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 30 15:46:58 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 15:46:58.954458 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6cfc33-1645-4020-a093-9e16ca0e55a2 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:58.955518 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9153c8-5d88-44d3-bd09-43c8f6d1f778 None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:58.955945 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6cfc33-1645-4020-a093-9e16ca0e55a2 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:58.969172 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-fd6cfc33-1645-4020-a093-9e16ca0e55a2 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 30 15:46:58.969172 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:46:58.969172 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:46:58.969172 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:46:58.969172 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:46:58.969172 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:46:58.969172 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:46:58.969172 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:46:58.969172 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:46:58.969172 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:46:58.969172 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:46:58.969172 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:46:58.969172 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 15:46:58.969172 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 15:46:58.969172 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 15:46:58.969172 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 15:46:58.970410 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:46:58.970410 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:46:58.970410 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 15:46:58.970410 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 15:46:58.970410 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 15:46:58.970410 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 15:46:58.970410 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:46:58.970410 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:46:58.970410 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:46:58.970410 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:46:58.970410 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:46:58.970410 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Aug 30 15:46:58.970410 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:46:58.970410 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:46:58.970410 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:46:58.970410 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:46:58.970410 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:46:58.970410 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:46:58.971546 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:46:58.971546 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:46:58.971546 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 15:46:58.971546 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 15:46:58.971546 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 15:46:58.971546 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 15:46:58.971546 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 15:46:58.971546 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 15:46:58.971546 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 30 15:46:58.971546 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Aug 30 15:46:58.971546 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 373/754] 10.222.0.20 () {56 vars in 1117 bytes} [Tue Aug 30 15:46:58 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 15:46:58.976413 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6cfc33-1645-4020-a093-9e16ca0e55a2 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:58.980306 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:59.068964 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9153c8-5d88-44d3-bd09-43c8f6d1f778 None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:59.078044 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6cfc33-1645-4020-a093-9e16ca0e55a2 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:59.106168 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3f945f-2eb8-400d-94af-a071a3f75544 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:59.107149 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3f945f-2eb8-400d-94af-a071a3f75544 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:59.107700 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3f945f-2eb8-400d-94af-a071a3f75544 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:59.108161 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3f945f-2eb8-400d-94af-a071a3f75544 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:59.112549 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1597d33b-6db7-40f2-9a52-b3587e7ebd68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:59.113165 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1597d33b-6db7-40f2-9a52-b3587e7ebd68 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:59.113499 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1597d33b-6db7-40f2-9a52-b3587e7ebd68 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:59.113769 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1597d33b-6db7-40f2-9a52-b3587e7ebd68 None admin] PATH_INFO: `/v3/users` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:59.114512 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1597d33b-6db7-40f2-9a52-b3587e7ebd68 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:46:59.114818 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1597d33b-6db7-40f2-9a52-b3587e7ebd68 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:59.116354 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1597d33b-6db7-40f2-9a52-b3587e7ebd68 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:59.117501 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3f945f-2eb8-400d-94af-a071a3f75544 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:59.119374 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3f945f-2eb8-400d-94af-a071a3f75544 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:59.132224 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application [None req-6f3f945f-2eb8-400d-94af-a071a3f75544 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 30 15:46:59.132224 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:46:59.132224 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:46:59.132224 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:46:59.132224 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:46:59.132224 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:46:59.132224 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:46:59.132224 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:46:59.132224 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:46:59.132224 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:46:59.132224 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:46:59.132224 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:46:59.132224 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 15:46:59.132224 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 15:46:59.132224 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 15:46:59.132224 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 15:46:59.133426 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:46:59.133426 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:46:59.133426 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 15:46:59.133426 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 15:46:59.133426 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 15:46:59.133426 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 15:46:59.133426 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:46:59.133426 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:46:59.133426 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:46:59.133426 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:46:59.133426 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:46:59.133426 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application with Lock( Aug 30 15:46:59.133426 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:46:59.133426 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:46:59.133426 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:46:59.133426 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:46:59.133426 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:46:59.133426 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:46:59.134608 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:46:59.134608 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:46:59.134608 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 15:46:59.134608 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 15:46:59.134608 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 15:46:59.134608 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 15:46:59.134608 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 15:46:59.134608 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 15:46:59.134608 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 30 15:46:59.134608 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Aug 30 15:46:59.135205 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 374/755] 10.222.0.20 () {56 vars in 1149 bytes} [Tue Aug 30 15:46:58 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 15:46:59.136770 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 382/756] 10.222.0.20 () {56 vars in 1093 bytes} [Tue Aug 30 15:46:58 2022] GET /identity/v3/users/nova => generated 82 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 15:46:59.140804 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1597d33b-6db7-40f2-9a52-b3587e7ebd68 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:59.142743 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3f945f-2eb8-400d-94af-a071a3f75544 None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:59.205559 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:59.209592 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:59.250113 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1597d33b-6db7-40f2-9a52-b3587e7ebd68 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:59.253903 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3f945f-2eb8-400d-94af-a071a3f75544 None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:59.288073 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f06568e-7c11-4c4c-acee-bf782ddcc20c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:59.288913 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f06568e-7c11-4c4c-acee-bf782ddcc20c None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:59.289364 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f06568e-7c11-4c4c-acee-bf782ddcc20c None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:59.289800 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f06568e-7c11-4c4c-acee-bf782ddcc20c None admin] PATH_INFO: `/v3/users` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:59.290677 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f06568e-7c11-4c4c-acee-bf782ddcc20c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81512) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:46:59.291141 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f06568e-7c11-4c4c-acee-bf782ddcc20c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:59.292715 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f06568e-7c11-4c4c-acee-bf782ddcc20c None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:59.309922 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e507a60e-7bdd-4f90-9175-70d536e90628 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:59.310540 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e507a60e-7bdd-4f90-9175-70d536e90628 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:59.310816 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e507a60e-7bdd-4f90-9175-70d536e90628 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:59.311141 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e507a60e-7bdd-4f90-9175-70d536e90628 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:59.311972 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 383/757] 10.222.0.20 () {56 vars in 1133 bytes} [Tue Aug 30 15:46:59 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 15:46:59.317491 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f06568e-7c11-4c4c-acee-bf782ddcc20c None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:59.379722 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e507a60e-7bdd-4f90-9175-70d536e90628 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:59.381305 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e507a60e-7bdd-4f90-9175-70d536e90628 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:59.442761 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:59.451091 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f06568e-7c11-4c4c-acee-bf782ddcc20c None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:59.470133 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:59.487079 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 375/758] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Aug 30 15:46:59 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 30 15:46:59.488025 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e507a60e-7bdd-4f90-9175-70d536e90628 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:59.489520 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2c4ecf-1369-419f-8c48-8e6374d404b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:59.490212 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2c4ecf-1369-419f-8c48-8e6374d404b8 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:59.490547 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2c4ecf-1369-419f-8c48-8e6374d404b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:59.490869 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2c4ecf-1369-419f-8c48-8e6374d404b8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:59.502117 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2c4ecf-1369-419f-8c48-8e6374d404b8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:59.503945 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2c4ecf-1369-419f-8c48-8e6374d404b8 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:59.520079 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application [None req-1d2c4ecf-1369-419f-8c48-8e6374d404b8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 15:46:59.520079 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:46:59.520079 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:46:59.520079 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:46:59.520079 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:46:59.520079 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:46:59.520079 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:46:59.520079 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:46:59.520079 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:46:59.520079 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:46:59.520079 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:46:59.520079 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:46:59.520079 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 15:46:59.520079 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 15:46:59.520079 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 15:46:59.520079 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 15:46:59.521286 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:46:59.521286 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:46:59.521286 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:46:59.521286 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:46:59.521286 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:46:59.521286 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:46:59.521286 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:46:59.521286 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application with Lock( Aug 30 15:46:59.521286 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:46:59.521286 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:46:59.521286 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:46:59.521286 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:46:59.521286 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:46:59.521286 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:46:59.521286 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:46:59.521286 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:46:59.521286 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 15:46:59.522265 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 15:46:59.522265 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 15:46:59.522265 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Aug 30 15:46:59.523438 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 384/759] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 30 15:46:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 15:46:59.524212 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2c4ecf-1369-419f-8c48-8e6374d404b8 None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:59.596260 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e507a60e-7bdd-4f90-9175-70d536e90628 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:59.632361 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0727edf0-6c41-4c5e-8da1-22abd1ae9a03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:59.633000 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0727edf0-6c41-4c5e-8da1-22abd1ae9a03 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:59.633311 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0727edf0-6c41-4c5e-8da1-22abd1ae9a03 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:59.633584 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0727edf0-6c41-4c5e-8da1-22abd1ae9a03 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:59.634176 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0727edf0-6c41-4c5e-8da1-22abd1ae9a03 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:59.635571 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0727edf0-6c41-4c5e-8da1-22abd1ae9a03 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:59.647243 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-0727edf0-6c41-4c5e-8da1-22abd1ae9a03 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 30 15:46:59.647243 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:46:59.647243 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 15:46:59.647243 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 15:46:59.647243 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 15:46:59.647243 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 15:46:59.647243 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 15:46:59.647243 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 15:46:59.647243 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 30 15:46:59.647243 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Aug 30 15:46:59.647243 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 15:46:59.647243 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Aug 30 15:46:59.647243 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:46:59.647243 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:46:59.647243 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:46:59.647243 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:46:59.647243 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:46:59.648289 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:46:59.648289 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:46:59.648289 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:46:59.648289 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:46:59.648289 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:46:59.648289 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:46:59.648289 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 15:46:59.648289 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 15:46:59.648289 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 15:46:59.648289 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 15:46:59.648289 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:46:59.648289 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:46:59.648289 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:46:59.648289 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:46:59.648289 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:46:59.648289 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:46:59.649134 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:46:59.649134 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Aug 30 15:46:59.649134 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:46:59.649134 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:46:59.649134 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:46:59.649134 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:46:59.649134 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:46:59.649134 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:46:59.649134 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:46:59.649134 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:46:59.649134 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 15:46:59.649134 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 15:46:59.649134 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 30 15:46:59.649134 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Aug 30 15:46:59.649134 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 376/760] 10.222.0.20 () {56 vars in 1129 bytes} [Tue Aug 30 15:46:59 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 15:46:59.652473 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0727edf0-6c41-4c5e-8da1-22abd1ae9a03 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:59.657988 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2c4ecf-1369-419f-8c48-8e6374d404b8 None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:59.681260 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:59.698741 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa61d00-e990-42c5-b4a3-877cb52a0cc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:59.699474 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa61d00-e990-42c5-b4a3-877cb52a0cc3 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:59.699620 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa61d00-e990-42c5-b4a3-877cb52a0cc3 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:59.699892 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa61d00-e990-42c5-b4a3-877cb52a0cc3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:59.711316 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa61d00-e990-42c5-b4a3-877cb52a0cc3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:59.712923 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa61d00-e990-42c5-b4a3-877cb52a0cc3 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:59.727919 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application [None req-2fa61d00-e990-42c5-b4a3-877cb52a0cc3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 15:46:59.727919 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:46:59.727919 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:46:59.727919 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:46:59.727919 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:46:59.727919 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:46:59.727919 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:46:59.727919 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:46:59.727919 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:46:59.727919 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:46:59.727919 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:46:59.727919 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:46:59.727919 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 15:46:59.727919 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 15:46:59.727919 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 15:46:59.727919 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 15:46:59.729172 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:46:59.729172 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:46:59.729172 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:46:59.729172 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:46:59.729172 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:46:59.729172 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:46:59.729172 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:46:59.729172 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application with Lock( Aug 30 15:46:59.729172 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:46:59.729172 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:46:59.729172 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:46:59.729172 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:46:59.729172 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:46:59.729172 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:46:59.729172 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:46:59.729172 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:46:59.729172 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 15:46:59.730186 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 15:46:59.730186 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 15:46:59.730186 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Aug 30 15:46:59.734614 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 385/761] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 30 15:46:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 15:46:59.734773 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa61d00-e990-42c5-b4a3-877cb52a0cc3 None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:59.760930 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:59.769390 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0727edf0-6c41-4c5e-8da1-22abd1ae9a03 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:59.807506 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd0b605-cb87-4c03-9aa6-7a7728d68623 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:59.808417 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd0b605-cb87-4c03-9aa6-7a7728d68623 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:59.808901 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd0b605-cb87-4c03-9aa6-7a7728d68623 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:59.809353 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd0b605-cb87-4c03-9aa6-7a7728d68623 None admin] PATH_INFO: `/v3/domains` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:59.810196 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd0b605-cb87-4c03-9aa6-7a7728d68623 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:46:59.810640 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd0b605-cb87-4c03-9aa6-7a7728d68623 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:59.812209 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd0b605-cb87-4c03-9aa6-7a7728d68623 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:59.825293 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 377/762] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Aug 30 15:46:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:46:59.826464 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd0b605-cb87-4c03-9aa6-7a7728d68623 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:59.840700 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa61d00-e990-42c5-b4a3-877cb52a0cc3 None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:59.875853 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa857d7-1640-4fda-bed1-8998d3614fe5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:59.876464 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa857d7-1640-4fda-bed1-8998d3614fe5 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:59.876737 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa857d7-1640-4fda-bed1-8998d3614fe5 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:59.877018 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa857d7-1640-4fda-bed1-8998d3614fe5 None admin] PATH_INFO: `/v3/services` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:59.877667 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa857d7-1640-4fda-bed1-8998d3614fe5 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81512) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:46:59.877966 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa857d7-1640-4fda-bed1-8998d3614fe5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:59.879332 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa857d7-1640-4fda-bed1-8998d3614fe5 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:59.888770 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 386/763] 10.222.0.20 () {56 vars in 1124 bytes} [Tue Aug 30 15:46:59 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:46:59.889912 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa857d7-1640-4fda-bed1-8998d3614fe5 None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:46:59.918028 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:59.935298 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd0b605-cb87-4c03-9aa6-7a7728d68623 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:46:59.971412 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5249fe99-41e6-43d2-aedc-a01d0c988495 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:46:59.972369 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5249fe99-41e6-43d2-aedc-a01d0c988495 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:46:59.972869 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5249fe99-41e6-43d2-aedc-a01d0c988495 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:46:59.973344 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5249fe99-41e6-43d2-aedc-a01d0c988495 None admin] PATH_INFO: `/v3/domains` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:46:59.974234 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5249fe99-41e6-43d2-aedc-a01d0c988495 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:46:59.974724 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5249fe99-41e6-43d2-aedc-a01d0c988495 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:46:59.976279 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5249fe99-41e6-43d2-aedc-a01d0c988495 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:46:59.978213 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:46:59.989425 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 378/764] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Aug 30 15:46:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:46:59.990598 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5249fe99-41e6-43d2-aedc-a01d0c988495 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:00.000444 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa857d7-1640-4fda-bed1-8998d3614fe5 None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:00.036080 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4add0a-983d-4829-93be-be03dfe070b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:00.036728 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4add0a-983d-4829-93be-be03dfe070b2 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:00.037001 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4add0a-983d-4829-93be-be03dfe070b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:00.037470 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4add0a-983d-4829-93be-be03dfe070b2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:00.044651 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4add0a-983d-4829-93be-be03dfe070b2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:00.046254 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4add0a-983d-4829-93be-be03dfe070b2 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:00.058157 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application [None req-0a4add0a-983d-4829-93be-be03dfe070b2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 15:47:00.058157 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:47:00.058157 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:47:00.058157 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:47:00.058157 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:47:00.058157 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:47:00.058157 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:47:00.058157 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:47:00.058157 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:47:00.058157 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:47:00.058157 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:47:00.058157 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:47:00.058157 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 15:47:00.058157 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 15:47:00.058157 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 15:47:00.059122 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 15:47:00.059122 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:47:00.059122 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:47:00.059122 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:47:00.059122 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:47:00.059122 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:47:00.059122 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:47:00.059122 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:47:00.059122 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application with Lock( Aug 30 15:47:00.059122 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:47:00.059122 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:47:00.059122 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:47:00.059122 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:47:00.059122 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:47:00.059122 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:47:00.059122 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:47:00.059122 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:47:00.060132 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 15:47:00.060132 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 15:47:00.060132 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 15:47:00.060132 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 15:47:00.060132 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 15:47:00.060132 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 15:47:00.060132 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 15:47:00.060132 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Aug 30 15:47:00.060132 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 387/765] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Aug 30 15:46:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 15:47:00.060671 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4add0a-983d-4829-93be-be03dfe070b2 None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:00.103988 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5249fe99-41e6-43d2-aedc-a01d0c988495 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:00.140544 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6550714f-a1cc-4bcf-97a2-4e73255fdc67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:00.141439 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6550714f-a1cc-4bcf-97a2-4e73255fdc67 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:00.141945 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6550714f-a1cc-4bcf-97a2-4e73255fdc67 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:00.142359 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6550714f-a1cc-4bcf-97a2-4e73255fdc67 None admin] PATH_INFO: `/v3/services` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:00.143136 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6550714f-a1cc-4bcf-97a2-4e73255fdc67 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:47:00.143580 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6550714f-a1cc-4bcf-97a2-4e73255fdc67 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:00.145441 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6550714f-a1cc-4bcf-97a2-4e73255fdc67 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:00.150701 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 379/766] 10.222.0.20 () {56 vars in 1124 bytes} [Tue Aug 30 15:46:59 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:47:00.151661 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6550714f-a1cc-4bcf-97a2-4e73255fdc67 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:00.164259 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:47:00.165093 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4add0a-983d-4829-93be-be03dfe070b2 None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:00.199570 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cddd46a-f2c9-4a54-9d29-89da6bfee17c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:00.200282 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cddd46a-f2c9-4a54-9d29-89da6bfee17c None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:00.200621 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cddd46a-f2c9-4a54-9d29-89da6bfee17c None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:00.200908 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cddd46a-f2c9-4a54-9d29-89da6bfee17c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:00.208445 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cddd46a-f2c9-4a54-9d29-89da6bfee17c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:00.209926 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cddd46a-f2c9-4a54-9d29-89da6bfee17c None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:00.221904 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application [None req-8cddd46a-f2c9-4a54-9d29-89da6bfee17c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 15:47:00.221904 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:47:00.221904 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:47:00.221904 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:47:00.221904 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:47:00.221904 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:47:00.221904 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:47:00.221904 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:47:00.221904 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:47:00.221904 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:47:00.221904 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:47:00.221904 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:47:00.221904 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 15:47:00.221904 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 15:47:00.221904 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 15:47:00.222916 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 15:47:00.222916 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:47:00.222916 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:47:00.222916 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:47:00.222916 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:47:00.222916 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:47:00.222916 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:47:00.222916 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:47:00.222916 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application with Lock( Aug 30 15:47:00.222916 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:47:00.222916 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:47:00.222916 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:47:00.222916 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:47:00.222916 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:47:00.222916 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:47:00.222916 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:47:00.222916 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:47:00.223806 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 15:47:00.223806 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 15:47:00.223806 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 15:47:00.223806 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 15:47:00.223806 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 15:47:00.223806 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 15:47:00.223806 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 15:47:00.223806 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Aug 30 15:47:00.223806 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 388/767] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Aug 30 15:47:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 15:47:00.224286 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11556379-53a4-4912-9401-2751b6ee00f3 None None] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:00.225276 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11556379-53a4-4912-9401-2751b6ee00f3 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:00.225276 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11556379-53a4-4912-9401-2751b6ee00f3 None None] PATH_INFO: `/` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:00.229388 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 389/768] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 30 15:47:00 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 15:47:00.229388 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11556379-53a4-4912-9401-2751b6ee00f3 None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:00.237107 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:47:00.264588 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6550714f-a1cc-4bcf-97a2-4e73255fdc67 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:00.300556 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096f1607-4d09-4aa6-a03c-d0bc8eba5eee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:00.301170 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096f1607-4d09-4aa6-a03c-d0bc8eba5eee None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:00.301493 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096f1607-4d09-4aa6-a03c-d0bc8eba5eee None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:00.301774 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096f1607-4d09-4aa6-a03c-d0bc8eba5eee None admin] PATH_INFO: `/v3/projects` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:00.302457 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096f1607-4d09-4aa6-a03c-d0bc8eba5eee None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:47:00.302794 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096f1607-4d09-4aa6-a03c-d0bc8eba5eee None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:00.304267 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096f1607-4d09-4aa6-a03c-d0bc8eba5eee None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:00.316369 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 380/769] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Aug 30 15:47:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:47:00.317250 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096f1607-4d09-4aa6-a03c-d0bc8eba5eee None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:00.353018 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11556379-53a4-4912-9401-2751b6ee00f3 None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:00.392931 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:47:00.393133 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c345a4f5-8430-4797-b659-e8cd36073752 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:00.393736 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c345a4f5-8430-4797-b659-e8cd36073752 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:00.394013 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c345a4f5-8430-4797-b659-e8cd36073752 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:00.394288 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c345a4f5-8430-4797-b659-e8cd36073752 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:00.394999 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c345a4f5-8430-4797-b659-e8cd36073752 None admin] RBAC: Adding query filter params (interface=public, service_id=8a906b9e200f4cdeb962337157f559b9, region_id=RegionOne) {{(pid=81512) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:47:00.395296 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c345a4f5-8430-4797-b659-e8cd36073752 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:00.396713 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c345a4f5-8430-4797-b659-e8cd36073752 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:00.404381 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 390/770] 10.222.0.20 () {56 vars in 1251 bytes} [Tue Aug 30 15:47:00 2022] GET /identity/v3/endpoints?service_id=8a906b9e200f4cdeb962337157f559b9&interface=public®ion_id=RegionOne => generated 194 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:47:00.405530 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357c8666-c98e-470f-a90b-cf8339e2fa5c None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:00.405816 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357c8666-c98e-470f-a90b-cf8339e2fa5c None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:00.406091 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357c8666-c98e-470f-a90b-cf8339e2fa5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:00.424791 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-357c8666-c98e-470f-a90b-cf8339e2fa5c None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:47:00.426004 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-357c8666-c98e-470f-a90b-cf8339e2fa5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:47:00.433087 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096f1607-4d09-4aa6-a03c-d0bc8eba5eee None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:00.471220 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63ed894-29c0-45fd-8c2c-2617a669cf6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:00.472072 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63ed894-29c0-45fd-8c2c-2617a669cf6a None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:00.472521 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63ed894-29c0-45fd-8c2c-2617a669cf6a None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:00.473098 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63ed894-29c0-45fd-8c2c-2617a669cf6a None admin] PATH_INFO: `/v3/projects` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:00.474055 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63ed894-29c0-45fd-8c2c-2617a669cf6a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:47:00.474581 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63ed894-29c0-45fd-8c2c-2617a669cf6a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:00.476320 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63ed894-29c0-45fd-8c2c-2617a669cf6a None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:00.489088 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:47:00.490145 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 381/771] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Aug 30 15:47:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:47:00.491314 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63ed894-29c0-45fd-8c2c-2617a669cf6a None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:00.499162 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 391/772] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 30 15:47:00 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 15:47:00.500006 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357c8666-c98e-470f-a90b-cf8339e2fa5c None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:00.602577 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63ed894-29c0-45fd-8c2c-2617a669cf6a None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:00.608072 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357c8666-c98e-470f-a90b-cf8339e2fa5c None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:00.637651 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:47:00.647276 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5412fd4-0d3d-4dce-9886-035a8fa09e7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:00.647988 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5412fd4-0d3d-4dce-9886-035a8fa09e7e None admin] REQUEST_METHOD: `PUT` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:00.648223 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5412fd4-0d3d-4dce-9886-035a8fa09e7e None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:00.648505 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5412fd4-0d3d-4dce-9886-035a8fa09e7e None admin] PATH_INFO: `/v3/projects/854fe71b49224dff9a4b1766a20ab266/users/7a1e756ae9be4a7bac1e81acf07e8f3b/roles/f265386950a641ac8413b53eda4ae34d` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:00.649146 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b916903-fbda-4d81-b661-af5d8b4d5cf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:00.649780 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b916903-fbda-4d81-b661-af5d8b4d5cf5 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:00.650054 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b916903-fbda-4d81-b661-af5d8b4d5cf5 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:00.650318 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b916903-fbda-4d81-b661-af5d8b4d5cf5 None admin] PATH_INFO: `/v3/services` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:00.650652 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5412fd4-0d3d-4dce-9886-035a8fa09e7e None admin] RBAC: Authorizing `identity:create_grant(project_id=854fe71b49224dff9a4b1766a20ab266, user_id=7a1e756ae9be4a7bac1e81acf07e8f3b, role_id=f265386950a641ac8413b53eda4ae34d)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:00.650917 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b916903-fbda-4d81-b661-af5d8b4d5cf5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:00.652083 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5412fd4-0d3d-4dce-9886-035a8fa09e7e None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:00.652270 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b916903-fbda-4d81-b661-af5d8b4d5cf5 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:00.658658 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 392/773] 10.222.0.20 () {56 vars in 1087 bytes} [Tue Aug 30 15:47:00 2022] GET /identity/v3/services => generated 772 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:47:00.659428 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b916903-fbda-4d81-b661-af5d8b4d5cf5 None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:00.682410 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 382/774] 10.222.0.20 () {58 vars in 1439 bytes} [Tue Aug 30 15:47:00 2022] PUT /identity/v3/projects/854fe71b49224dff9a4b1766a20ab266/users/7a1e756ae9be4a7bac1e81acf07e8f3b/roles/f265386950a641ac8413b53eda4ae34d => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:47:00.683720 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc837ff2-d0df-4123-ae2c-bd7036d3d3a7 None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:00.683991 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc837ff2-d0df-4123-ae2c-bd7036d3d3a7 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:00.684259 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc837ff2-d0df-4123-ae2c-bd7036d3d3a7 None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:00.685735 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 383/775] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 30 15:47:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 15:47:00.691341 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8bcd22-80be-4ed9-b1ad-1ed635d1d059 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:00.691696 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8bcd22-80be-4ed9-b1ad-1ed635d1d059 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:00.691964 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8bcd22-80be-4ed9-b1ad-1ed635d1d059 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:00.711714 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-7a8bcd22-80be-4ed9-b1ad-1ed635d1d059 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:47:00.712944 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-7a8bcd22-80be-4ed9-b1ad-1ed635d1d059 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:47:00.752233 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:47:00.795135 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 384/776] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 30 15:47:00 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 30 15:47:00.796095 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8bcd22-80be-4ed9-b1ad-1ed635d1d059 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:00.809753 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b916903-fbda-4d81-b661-af5d8b4d5cf5 None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:00.856589 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22afa01e-4e2e-4aab-a0e8-f146c745dc43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:00.857721 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22afa01e-4e2e-4aab-a0e8-f146c745dc43 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:00.858957 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22afa01e-4e2e-4aab-a0e8-f146c745dc43 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:00.859473 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22afa01e-4e2e-4aab-a0e8-f146c745dc43 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:00.860718 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22afa01e-4e2e-4aab-a0e8-f146c745dc43 None admin] RBAC: Adding query filter params (role.id=57eb8466096346b39319536c6feeb6f7, scope.project.id=854fe71b49224dff9a4b1766a20ab266, user.id=07130d598131443486c2bc1c6673c45b) {{(pid=81512) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:47:00.861589 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22afa01e-4e2e-4aab-a0e8-f146c745dc43 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:00.863948 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22afa01e-4e2e-4aab-a0e8-f146c745dc43 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:00.875176 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 393/777] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Aug 30 15:47:00 2022] GET /identity/v3/role_assignments?user.id=07130d598131443486c2bc1c6673c45b&scope.project.id=854fe71b49224dff9a4b1766a20ab266&role.id=57eb8466096346b39319536c6feeb6f7 => generated 259 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:47:00.949558 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8bcd22-80be-4ed9-b1ad-1ed635d1d059 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:00.974116 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:47:00.990501 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8298ce4-8613-4ef7-9a58-66a69799f8f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:00.991153 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8298ce4-8613-4ef7-9a58-66a69799f8f0 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:00.991429 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8298ce4-8613-4ef7-9a58-66a69799f8f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:00.991694 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8298ce4-8613-4ef7-9a58-66a69799f8f0 None admin] PATH_INFO: `/v3/services/network` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:00.992296 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8298ce4-8613-4ef7-9a58-66a69799f8f0 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:00.993732 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8298ce4-8613-4ef7-9a58-66a69799f8f0 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:01.003864 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-d8298ce4-8613-4ef7-9a58-66a69799f8f0 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 30 15:47:01.003864 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:47:01.003864 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 15:47:01.003864 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 15:47:01.003864 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 15:47:01.003864 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 15:47:01.003864 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 15:47:01.003864 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 15:47:01.003864 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 30 15:47:01.003864 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Aug 30 15:47:01.003864 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 15:47:01.003864 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Aug 30 15:47:01.003864 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:47:01.003864 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:47:01.003864 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:47:01.003864 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:47:01.003864 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:47:01.004914 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:47:01.004914 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:47:01.004914 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:47:01.004914 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:47:01.004914 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:47:01.004914 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:47:01.004914 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 15:47:01.004914 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 15:47:01.004914 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 15:47:01.004914 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 15:47:01.004914 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:47:01.004914 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:47:01.004914 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:47:01.004914 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:47:01.004914 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:47:01.004914 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:47:01.005815 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:47:01.005815 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Aug 30 15:47:01.005815 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:47:01.005815 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:47:01.005815 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:47:01.005815 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:47:01.005815 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:47:01.005815 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:47:01.005815 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:47:01.005815 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:47:01.005815 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 15:47:01.005815 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 15:47:01.005815 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 30 15:47:01.005815 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Aug 30 15:47:01.005815 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 385/778] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Aug 30 15:47:00 2022] GET /identity/v3/services/network => generated 88 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 15:47:01.011133 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22afa01e-4e2e-4aab-a0e8-f146c745dc43 None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:01.111542 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8298ce4-8613-4ef7-9a58-66a69799f8f0 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:01.187159 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22afa01e-4e2e-4aab-a0e8-f146c745dc43 None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:01.236693 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:47:01.252746 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22df21a3-4d47-472d-abf5-a0fe21e8a28a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:01.253397 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22df21a3-4d47-472d-abf5-a0fe21e8a28a None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:01.253690 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22df21a3-4d47-472d-abf5-a0fe21e8a28a None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:01.254118 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22df21a3-4d47-472d-abf5-a0fe21e8a28a None admin] PATH_INFO: `/v3/services` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:01.254701 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22df21a3-4d47-472d-abf5-a0fe21e8a28a None admin] RBAC: Adding query filter params (name=network) {{(pid=81512) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:47:01.254996 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22df21a3-4d47-472d-abf5-a0fe21e8a28a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:01.257709 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22df21a3-4d47-472d-abf5-a0fe21e8a28a None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:01.267683 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 394/779] 10.222.0.20 () {56 vars in 1112 bytes} [Tue Aug 30 15:47:01 2022] GET /identity/v3/services?name=network => generated 124 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:47:01.273246 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22df21a3-4d47-472d-abf5-a0fe21e8a28a None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:01.283465 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8298ce4-8613-4ef7-9a58-66a69799f8f0 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:01.327115 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028fa173-7c41-47bf-ac6c-dc5283b5b479 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:01.328264 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028fa173-7c41-47bf-ac6c-dc5283b5b479 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:01.328771 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028fa173-7c41-47bf-ac6c-dc5283b5b479 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:01.329308 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028fa173-7c41-47bf-ac6c-dc5283b5b479 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:01.342751 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028fa173-7c41-47bf-ac6c-dc5283b5b479 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 15:47:01.348920 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-028fa173-7c41-47bf-ac6c-dc5283b5b479 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 15:47:01.348920 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:47:01.348920 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:47:01.348920 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:47:01.348920 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:47:01.348920 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:47:01.348920 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:47:01.348920 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:47:01.348920 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:47:01.348920 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:47:01.348920 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:47:01.348920 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:47:01.348920 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 15:47:01.348920 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 15:47:01.348920 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 15:47:01.348920 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 15:47:01.349942 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 15:47:01.349942 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 15:47:01.349942 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 15:47:01.349942 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 15:47:01.349942 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:47:01.349942 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:47:01.349942 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:47:01.349942 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:47:01.349942 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:47:01.349942 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:47:01.349942 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:47:01.349942 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Aug 30 15:47:01.349942 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:47:01.349942 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:47:01.349942 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:47:01.349942 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:47:01.350800 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:47:01.350800 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:47:01.350800 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:47:01.350800 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:47:01.350800 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 15:47:01.350800 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 15:47:01.350800 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 15:47:01.350800 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 15:47:01.350800 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 15:47:01.350800 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 15:47:01.350800 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 15:47:01.350800 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Aug 30 15:47:01.353532 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 386/780] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Aug 30 15:47:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 15:47:01.365437 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028fa173-7c41-47bf-ac6c-dc5283b5b479 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:01.409618 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22df21a3-4d47-472d-abf5-a0fe21e8a28a None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:01.456826 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:47:01.465077 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cb0085-1864-4e02-89d7-a79701095808 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:01.465722 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cb0085-1864-4e02-89d7-a79701095808 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:01.465998 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cb0085-1864-4e02-89d7-a79701095808 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:01.466266 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cb0085-1864-4e02-89d7-a79701095808 None admin] PATH_INFO: `/v3/services` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:01.466896 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cb0085-1864-4e02-89d7-a79701095808 None admin] RBAC: Adding query filter params (type=network) {{(pid=81512) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:47:01.467272 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cb0085-1864-4e02-89d7-a79701095808 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:01.470517 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cb0085-1864-4e02-89d7-a79701095808 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:01.481244 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 395/781] 10.222.0.20 () {56 vars in 1112 bytes} [Tue Aug 30 15:47:01 2022] GET /identity/v3/services?type=network => generated 350 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 15:47:01.487281 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cb0085-1864-4e02-89d7-a79701095808 None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:01.565435 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028fa173-7c41-47bf-ac6c-dc5283b5b479 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:01.594040 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:47:01.636404 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fbda8c-14c9-43aa-9e48-66cc589326fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:01.637021 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fbda8c-14c9-43aa-9e48-66cc589326fc None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:01.637316 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fbda8c-14c9-43aa-9e48-66cc589326fc None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:01.637590 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fbda8c-14c9-43aa-9e48-66cc589326fc None admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:01.638257 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fbda8c-14c9-43aa-9e48-66cc589326fc None admin] RBAC: Adding query filter params (name=admin) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:47:01.638557 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fbda8c-14c9-43aa-9e48-66cc589326fc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:01.640409 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fbda8c-14c9-43aa-9e48-66cc589326fc None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:01.660893 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 387/782] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Aug 30 15:47:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:47:01.666699 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fbda8c-14c9-43aa-9e48-66cc589326fc None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:01.688554 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cb0085-1864-4e02-89d7-a79701095808 None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:01.756185 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcd1caf-e3e2-4ff1-bf8d-198d77ff3626 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:01.759142 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcd1caf-e3e2-4ff1-bf8d-198d77ff3626 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:01.759641 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcd1caf-e3e2-4ff1-bf8d-198d77ff3626 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:01.762203 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcd1caf-e3e2-4ff1-bf8d-198d77ff3626 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:01.763133 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcd1caf-e3e2-4ff1-bf8d-198d77ff3626 None admin] RBAC: Adding query filter params (interface=public, service_id=f5470b40938b440d971910de9c529bfc, region_id=RegionOne) {{(pid=81512) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:47:01.763660 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcd1caf-e3e2-4ff1-bf8d-198d77ff3626 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:01.766544 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcd1caf-e3e2-4ff1-bf8d-198d77ff3626 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:01.778057 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 396/783] 10.222.0.20 () {56 vars in 1251 bytes} [Tue Aug 30 15:47:01 2022] GET /identity/v3/endpoints?service_id=f5470b40938b440d971910de9c529bfc&interface=public®ion_id=RegionOne => generated 194 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:47:01.784456 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcd1caf-e3e2-4ff1-bf8d-198d77ff3626 None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:01.861100 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fbda8c-14c9-43aa-9e48-66cc589326fc None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:01.871435 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:47:01.931001 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c2b555-8305-48ec-acf4-6f447644549b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:01.932032 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c2b555-8305-48ec-acf4-6f447644549b None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:01.932501 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c2b555-8305-48ec-acf4-6f447644549b None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:01.932981 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c2b555-8305-48ec-acf4-6f447644549b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:01.962873 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c2b555-8305-48ec-acf4-6f447644549b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:01.964752 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c2b555-8305-48ec-acf4-6f447644549b None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:01.999776 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-79c2b555-8305-48ec-acf4-6f447644549b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 15:47:01.999776 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:47:01.999776 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:47:01.999776 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:47:01.999776 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:47:01.999776 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:47:01.999776 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:47:01.999776 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:47:01.999776 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:47:01.999776 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:47:01.999776 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:47:01.999776 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:47:01.999776 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 15:47:01.999776 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 15:47:01.999776 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 15:47:01.999776 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 15:47:02.000815 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:47:02.000815 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:47:02.000815 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:47:02.000815 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:47:02.000815 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:47:02.000815 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:47:02.000815 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:47:02.000815 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Aug 30 15:47:02.000815 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:47:02.000815 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:47:02.000815 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:47:02.000815 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:47:02.000815 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:47:02.000815 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:47:02.000815 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:47:02.000815 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:47:02.000815 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 15:47:02.001742 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 15:47:02.001742 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 15:47:02.001742 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Aug 30 15:47:02.003989 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 388/784] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 30 15:47:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 337 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 15:47:02.010371 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c2b555-8305-48ec-acf4-6f447644549b None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:02.115036 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcd1caf-e3e2-4ff1-bf8d-198d77ff3626 None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:02.144981 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c2b555-8305-48ec-acf4-6f447644549b None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:02.145161 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:47:02.157469 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d3f857-d86d-4099-9b57-0d7db86f6a94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:02.158142 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d3f857-d86d-4099-9b57-0d7db86f6a94 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:02.158419 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d3f857-d86d-4099-9b57-0d7db86f6a94 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:02.158685 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d3f857-d86d-4099-9b57-0d7db86f6a94 None admin] PATH_INFO: `/v3/services` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:02.159343 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d3f857-d86d-4099-9b57-0d7db86f6a94 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:02.163961 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d3f857-d86d-4099-9b57-0d7db86f6a94 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:02.167335 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 397/785] 10.222.0.20 () {56 vars in 1087 bytes} [Tue Aug 30 15:47:01 2022] GET /identity/v3/services => generated 772 bytes in 384 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 15:47:02.207335 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b479e42e-a5f6-4186-847b-b319e8d86fbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:02.208403 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b479e42e-a5f6-4186-847b-b319e8d86fbf None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:02.208889 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b479e42e-a5f6-4186-847b-b319e8d86fbf None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:02.209472 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b479e42e-a5f6-4186-847b-b319e8d86fbf None admin] PATH_INFO: `/v3/domains` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:02.210343 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b479e42e-a5f6-4186-847b-b319e8d86fbf None admin] RBAC: Adding query filter params (name=Default) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:47:02.210843 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b479e42e-a5f6-4186-847b-b319e8d86fbf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:02.212558 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b479e42e-a5f6-4186-847b-b319e8d86fbf None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:02.232294 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 389/786] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Aug 30 15:47:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:47:02.238277 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d3f857-d86d-4099-9b57-0d7db86f6a94 None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:02.374955 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d3f857-d86d-4099-9b57-0d7db86f6a94 None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:02.411882 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb54e733-8bae-4701-8b47-9287d947721b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:02.412526 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb54e733-8bae-4701-8b47-9287d947721b None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:02.412797 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb54e733-8bae-4701-8b47-9287d947721b None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:02.413064 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb54e733-8bae-4701-8b47-9287d947721b None admin] PATH_INFO: `/v3/users/swift` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:02.421499 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb54e733-8bae-4701-8b47-9287d947721b None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:02.423021 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb54e733-8bae-4701-8b47-9287d947721b None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:02.434683 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application [None req-fb54e733-8bae-4701-8b47-9287d947721b None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 30 15:47:02.434683 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:47:02.434683 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:47:02.434683 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:47:02.434683 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:47:02.434683 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:47:02.434683 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:47:02.434683 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:47:02.434683 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:47:02.434683 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:47:02.434683 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:47:02.434683 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:47:02.434683 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 15:47:02.434683 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 15:47:02.434683 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 15:47:02.434683 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 15:47:02.435647 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:47:02.435647 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:47:02.435647 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 15:47:02.435647 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 15:47:02.435647 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 15:47:02.435647 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 15:47:02.435647 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:47:02.435647 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:47:02.435647 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:47:02.435647 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:47:02.435647 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:47:02.435647 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application with Lock( Aug 30 15:47:02.435647 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:47:02.435647 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:47:02.435647 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:47:02.435647 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:47:02.435647 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:47:02.435647 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:47:02.436676 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:47:02.436676 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:47:02.436676 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 15:47:02.436676 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 15:47:02.436676 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 15:47:02.436676 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 15:47:02.436676 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 15:47:02.436676 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 15:47:02.436676 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 30 15:47:02.436676 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Aug 30 15:47:02.436676 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 398/787] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Aug 30 15:47:02 2022] GET /identity/v3/users/swift => generated 83 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 15:47:02.443611 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b479e42e-a5f6-4186-847b-b319e8d86fbf None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:02.463726 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:47:02.569295 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b479e42e-a5f6-4186-847b-b319e8d86fbf None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:02.606098 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b87c7ac-f1a0-4afe-bc82-738dc9dcf974 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:02.606706 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b87c7ac-f1a0-4afe-bc82-738dc9dcf974 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:02.606983 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b87c7ac-f1a0-4afe-bc82-738dc9dcf974 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:02.607253 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b87c7ac-f1a0-4afe-bc82-738dc9dcf974 None admin] PATH_INFO: `/v3/users` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:02.608047 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b87c7ac-f1a0-4afe-bc82-738dc9dcf974 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:47:02.608349 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b87c7ac-f1a0-4afe-bc82-738dc9dcf974 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:02.609817 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b87c7ac-f1a0-4afe-bc82-738dc9dcf974 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:02.628272 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 390/788] 10.222.0.20 () {56 vars in 1135 bytes} [Tue Aug 30 15:47:02 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:47:02.633597 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb54e733-8bae-4701-8b47-9287d947721b None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:02.682906 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:47:02.740141 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb54e733-8bae-4701-8b47-9287d947721b None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:02.775335 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced1642b-89a8-4d06-ba81-ecb2b412edfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:02.775965 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced1642b-89a8-4d06-ba81-ecb2b412edfe None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:02.776238 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced1642b-89a8-4d06-ba81-ecb2b412edfe None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:02.776504 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced1642b-89a8-4d06-ba81-ecb2b412edfe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:02.787548 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced1642b-89a8-4d06-ba81-ecb2b412edfe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:02.789088 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced1642b-89a8-4d06-ba81-ecb2b412edfe None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:02.803805 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application [None req-ced1642b-89a8-4d06-ba81-ecb2b412edfe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 15:47:02.803805 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:47:02.803805 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:47:02.803805 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:47:02.803805 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:47:02.803805 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:47:02.803805 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:47:02.803805 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:47:02.803805 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:47:02.803805 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:47:02.803805 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:47:02.803805 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:47:02.803805 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 15:47:02.803805 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 15:47:02.803805 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 15:47:02.803805 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 15:47:02.804817 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:47:02.804817 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:47:02.804817 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:47:02.804817 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:47:02.804817 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:47:02.804817 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:47:02.804817 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:47:02.804817 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application with Lock( Aug 30 15:47:02.804817 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:47:02.804817 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:47:02.804817 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:47:02.804817 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:47:02.804817 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:47:02.804817 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:47:02.804817 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:47:02.804817 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:47:02.804817 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 15:47:02.805731 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 15:47:02.805731 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 15:47:02.805731 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Aug 30 15:47:02.805731 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 399/789] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 30 15:47:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 15:47:02.814413 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b87c7ac-f1a0-4afe-bc82-738dc9dcf974 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:02.871262 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a58294-4bcd-4599-8535-641b418255ab None None] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:02.871599 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a58294-4bcd-4599-8535-641b418255ab None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:02.871815 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a58294-4bcd-4599-8535-641b418255ab None None] PATH_INFO: `/` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:02.873468 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 400/790] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 30 15:47:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 15:47:02.878408 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abe6468-ed74-4321-b8b9-74e9810fd08d None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:02.878622 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abe6468-ed74-4321-b8b9-74e9810fd08d None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:02.878897 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abe6468-ed74-4321-b8b9-74e9810fd08d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:02.897416 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-0abe6468-ed74-4321-b8b9-74e9810fd08d None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:47:02.898612 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-0abe6468-ed74-4321-b8b9-74e9810fd08d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:47:02.932861 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:47:02.987151 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b87c7ac-f1a0-4afe-bc82-738dc9dcf974 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:02.992153 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 401/791] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 30 15:47:02 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 30 15:47:02.998705 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca20a3d-05fd-40b6-8055-9d2e3620d34e None None] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:02.999248 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca20a3d-05fd-40b6-8055-9d2e3620d34e None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:02.999719 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca20a3d-05fd-40b6-8055-9d2e3620d34e None None] PATH_INFO: `/` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:03.001534 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 402/792] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 30 15:47:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 15:47:03.007507 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0101f76e-9ad6-4704-8ea5-e9d3a83d9cfc None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:03.008023 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0101f76e-9ad6-4704-8ea5-e9d3a83d9cfc None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:03.008495 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0101f76e-9ad6-4704-8ea5-e9d3a83d9cfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:03.033240 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17779b35-b976-4b23-8799-67c63eddd5bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:03.033240 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17779b35-b976-4b23-8799-67c63eddd5bf None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:03.033240 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17779b35-b976-4b23-8799-67c63eddd5bf None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:03.033240 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17779b35-b976-4b23-8799-67c63eddd5bf None admin] PATH_INFO: `/v3/domains` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:03.033662 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17779b35-b976-4b23-8799-67c63eddd5bf None admin] RBAC: Adding query filter params (name=Default) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:47:03.033662 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17779b35-b976-4b23-8799-67c63eddd5bf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:03.033662 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17779b35-b976-4b23-8799-67c63eddd5bf None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:03.040045 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-0101f76e-9ad6-4704-8ea5-e9d3a83d9cfc None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:47:03.041460 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-0101f76e-9ad6-4704-8ea5-e9d3a83d9cfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:47:03.049307 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 391/793] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Aug 30 15:47:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:47:03.049559 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17779b35-b976-4b23-8799-67c63eddd5bf None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:03.089276 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:47:03.160422 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17779b35-b976-4b23-8799-67c63eddd5bf None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:03.197291 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 403/794] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 30 15:47:03 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 30 15:47:03.220703 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3212156-0d38-445d-9a8b-d3b27c4bc3be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:03.221391 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3212156-0d38-445d-9a8b-d3b27c4bc3be None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:03.221673 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3212156-0d38-445d-9a8b-d3b27c4bc3be None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:03.221948 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3212156-0d38-445d-9a8b-d3b27c4bc3be None admin] PATH_INFO: `/v3/projects/service` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:03.228925 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3212156-0d38-445d-9a8b-d3b27c4bc3be None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:03.230428 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3212156-0d38-445d-9a8b-d3b27c4bc3be None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:03.231097 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:47:03.249965 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-a3212156-0d38-445d-9a8b-d3b27c4bc3be None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 15:47:03.249965 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:47:03.249965 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:47:03.249965 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:47:03.249965 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:47:03.249965 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:47:03.249965 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:47:03.249965 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:47:03.249965 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:47:03.249965 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:47:03.249965 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:47:03.249965 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:47:03.249965 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 15:47:03.249965 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 15:47:03.249965 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 15:47:03.250927 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 15:47:03.250927 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:47:03.250927 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:47:03.250927 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:47:03.250927 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:47:03.250927 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:47:03.250927 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:47:03.250927 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:47:03.250927 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Aug 30 15:47:03.250927 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:47:03.250927 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:47:03.250927 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:47:03.250927 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:47:03.250927 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:47:03.250927 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:47:03.250927 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:47:03.250927 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:47:03.251846 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 15:47:03.251846 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 15:47:03.251846 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 15:47:03.251846 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 15:47:03.251846 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 15:47:03.251846 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 15:47:03.251846 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 15:47:03.251846 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Aug 30 15:47:03.251846 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 392/795] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Aug 30 15:47:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 15:47:03.257330 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0101f76e-9ad6-4704-8ea5-e9d3a83d9cfc None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:03.332928 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3212156-0d38-445d-9a8b-d3b27c4bc3be None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:03.419361 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0101f76e-9ad6-4704-8ea5-e9d3a83d9cfc None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:03.454780 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321d2c11-ccb7-4cd4-9ec6-03e9dd9f2b23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:03.455401 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321d2c11-ccb7-4cd4-9ec6-03e9dd9f2b23 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:03.455676 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321d2c11-ccb7-4cd4-9ec6-03e9dd9f2b23 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:03.455948 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321d2c11-ccb7-4cd4-9ec6-03e9dd9f2b23 None admin] PATH_INFO: `/v3/projects` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:03.456744 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321d2c11-ccb7-4cd4-9ec6-03e9dd9f2b23 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81512) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:47:03.457054 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321d2c11-ccb7-4cd4-9ec6-03e9dd9f2b23 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:03.458529 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321d2c11-ccb7-4cd4-9ec6-03e9dd9f2b23 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:03.468853 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3212156-0d38-445d-9a8b-d3b27c4bc3be None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:03.476186 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 404/796] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Aug 30 15:47:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:47:03.477566 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e62c58f-2e2a-4f43-9ffc-2860bf3e4b49 None None] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:03.477852 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e62c58f-2e2a-4f43-9ffc-2860bf3e4b49 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:03.478124 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e62c58f-2e2a-4f43-9ffc-2860bf3e4b49 None None] PATH_INFO: `/` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:03.479937 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 405/797] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 30 15:47:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 15:47:03.480823 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e62c58f-2e2a-4f43-9ffc-2860bf3e4b49 None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:03.501021 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:47:03.510416 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddede21-957a-49af-92b5-44ec26a29252 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:03.511058 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddede21-957a-49af-92b5-44ec26a29252 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:03.511336 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddede21-957a-49af-92b5-44ec26a29252 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:03.511606 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddede21-957a-49af-92b5-44ec26a29252 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:03.512199 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddede21-957a-49af-92b5-44ec26a29252 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:03.513604 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddede21-957a-49af-92b5-44ec26a29252 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:03.522386 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-8ddede21-957a-49af-92b5-44ec26a29252 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 30 15:47:03.522386 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:47:03.522386 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 15:47:03.522386 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 15:47:03.522386 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 15:47:03.522386 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 15:47:03.522386 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 15:47:03.522386 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 15:47:03.522386 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 30 15:47:03.522386 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Aug 30 15:47:03.522386 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 15:47:03.522386 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Aug 30 15:47:03.522386 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:47:03.522386 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:47:03.522386 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:47:03.522386 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:47:03.522386 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:47:03.523389 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:47:03.523389 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:47:03.523389 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:47:03.523389 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:47:03.523389 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:47:03.523389 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:47:03.523389 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 15:47:03.523389 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 15:47:03.523389 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 15:47:03.523389 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 15:47:03.523389 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:47:03.523389 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:47:03.523389 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:47:03.523389 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:47:03.523389 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:47:03.523389 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:47:03.524246 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:47:03.524246 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Aug 30 15:47:03.524246 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:47:03.524246 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:47:03.524246 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:47:03.524246 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:47:03.524246 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:47:03.524246 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:47:03.524246 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:47:03.524246 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:47:03.524246 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 15:47:03.524246 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 15:47:03.524246 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 30 15:47:03.524246 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Aug 30 15:47:03.524246 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 393/798] 10.222.0.20 () {56 vars in 1129 bytes} [Tue Aug 30 15:47:03 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 15:47:03.525060 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8baf0987-2c0a-41b4-9dd4-2ebfb0528112 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:03.525667 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8baf0987-2c0a-41b4-9dd4-2ebfb0528112 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:03.525667 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8baf0987-2c0a-41b4-9dd4-2ebfb0528112 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:03.543806 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-8baf0987-2c0a-41b4-9dd4-2ebfb0528112 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:47:03.545009 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-8baf0987-2c0a-41b4-9dd4-2ebfb0528112 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:47:03.591261 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e62c58f-2e2a-4f43-9ffc-2860bf3e4b49 None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:03.602194 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:47:03.614829 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 394/799] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 30 15:47:03 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 15:47:03.616100 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8baf0987-2c0a-41b4-9dd4-2ebfb0528112 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:03.652305 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b167e8-7502-4f85-b107-b4b9b69a2537 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:03.653306 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b167e8-7502-4f85-b107-b4b9b69a2537 None admin] REQUEST_METHOD: `PUT` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:03.653759 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b167e8-7502-4f85-b107-b4b9b69a2537 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:03.654259 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b167e8-7502-4f85-b107-b4b9b69a2537 None admin] PATH_INFO: `/v3/projects/854fe71b49224dff9a4b1766a20ab266/users/8fdafdc39a594bb4b0aeaf2fbc84c2fb/roles/f265386950a641ac8413b53eda4ae34d` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:03.656706 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b167e8-7502-4f85-b107-b4b9b69a2537 None admin] RBAC: Authorizing `identity:create_grant(project_id=854fe71b49224dff9a4b1766a20ab266, user_id=8fdafdc39a594bb4b0aeaf2fbc84c2fb, role_id=f265386950a641ac8413b53eda4ae34d)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:03.658968 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b167e8-7502-4f85-b107-b4b9b69a2537 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:03.676149 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 406/800] 10.222.0.20 () {58 vars in 1439 bytes} [Tue Aug 30 15:47:03 2022] PUT /identity/v3/projects/854fe71b49224dff9a4b1766a20ab266/users/8fdafdc39a594bb4b0aeaf2fbc84c2fb/roles/f265386950a641ac8413b53eda4ae34d => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 15:47:03.677726 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b167e8-7502-4f85-b107-b4b9b69a2537 None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:03.732735 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8baf0987-2c0a-41b4-9dd4-2ebfb0528112 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:03.771501 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:47:03.775990 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48707db9-3d54-4ff6-bf71-813f5f770bcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:03.776701 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48707db9-3d54-4ff6-bf71-813f5f770bcf None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:03.776985 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48707db9-3d54-4ff6-bf71-813f5f770bcf None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:03.777279 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48707db9-3d54-4ff6-bf71-813f5f770bcf None admin] PATH_INFO: `/v3/services` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:03.778052 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48707db9-3d54-4ff6-bf71-813f5f770bcf None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:47:03.778365 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48707db9-3d54-4ff6-bf71-813f5f770bcf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:03.781199 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48707db9-3d54-4ff6-bf71-813f5f770bcf None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:03.788725 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 395/801] 10.222.0.20 () {56 vars in 1124 bytes} [Tue Aug 30 15:47:03 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:47:03.795157 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48707db9-3d54-4ff6-bf71-813f5f770bcf None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:03.808644 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b167e8-7502-4f85-b107-b4b9b69a2537 None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:03.858871 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:47:03.861486 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c41c44d-baa5-48ed-8464-0c38a0b35e53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:03.863283 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c41c44d-baa5-48ed-8464-0c38a0b35e53 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:03.863283 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c41c44d-baa5-48ed-8464-0c38a0b35e53 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:03.863283 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c41c44d-baa5-48ed-8464-0c38a0b35e53 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:03.887501 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c41c44d-baa5-48ed-8464-0c38a0b35e53 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 15:47:03.899590 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application [None req-7c41c44d-baa5-48ed-8464-0c38a0b35e53 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 15:47:03.899590 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:47:03.899590 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:47:03.899590 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:47:03.899590 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:47:03.899590 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:47:03.899590 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:47:03.899590 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:47:03.899590 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:47:03.899590 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:47:03.899590 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:47:03.899590 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:47:03.899590 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 15:47:03.899590 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 15:47:03.899590 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 15:47:03.899590 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 15:47:03.900599 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 15:47:03.900599 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 15:47:03.900599 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 15:47:03.900599 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 15:47:03.900599 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:47:03.900599 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:47:03.900599 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:47:03.900599 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:47:03.900599 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:47:03.900599 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:47:03.900599 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:47:03.900599 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application with Lock( Aug 30 15:47:03.900599 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:47:03.900599 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:47:03.900599 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:47:03.900599 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:47:03.901478 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:47:03.901478 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:47:03.901478 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:47:03.901478 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:47:03.901478 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 15:47:03.901478 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 15:47:03.901478 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 15:47:03.901478 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 15:47:03.901478 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 15:47:03.901478 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 15:47:03.901478 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 15:47:03.901478 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Aug 30 15:47:03.903863 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 407/802] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Aug 30 15:47:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 15:47:03.909960 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c41c44d-baa5-48ed-8464-0c38a0b35e53 None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:03.910956 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48707db9-3d54-4ff6-bf71-813f5f770bcf None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:03.947703 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07482956-d99b-43b8-874a-6575a2183e08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:03.948232 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07482956-d99b-43b8-874a-6575a2183e08 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:03.948512 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07482956-d99b-43b8-874a-6575a2183e08 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:03.948784 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07482956-d99b-43b8-874a-6575a2183e08 None admin] PATH_INFO: `/v3/services` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:03.949464 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07482956-d99b-43b8-874a-6575a2183e08 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:47:03.949768 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07482956-d99b-43b8-874a-6575a2183e08 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:03.951151 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07482956-d99b-43b8-874a-6575a2183e08 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:03.956251 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 396/803] 10.222.0.20 () {56 vars in 1124 bytes} [Tue Aug 30 15:47:03 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 15:47:03.956884 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07482956-d99b-43b8-874a-6575a2183e08 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:04.023376 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c41c44d-baa5-48ed-8464-0c38a0b35e53 None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:04.058411 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d7da1e-b254-4ae7-9670-21fe71221e89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:04.059469 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d7da1e-b254-4ae7-9670-21fe71221e89 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:04.059943 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d7da1e-b254-4ae7-9670-21fe71221e89 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:04.060420 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d7da1e-b254-4ae7-9670-21fe71221e89 None admin] PATH_INFO: `/v3/roles` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:04.061337 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d7da1e-b254-4ae7-9670-21fe71221e89 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81512) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:47:04.061851 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d7da1e-b254-4ae7-9670-21fe71221e89 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:04.062229 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07482956-d99b-43b8-874a-6575a2183e08 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:04.063716 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d7da1e-b254-4ae7-9670-21fe71221e89 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:04.066292 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:47:04.075397 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 408/804] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Aug 30 15:47:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 15:47:04.076367 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d7da1e-b254-4ae7-9670-21fe71221e89 None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:04.089667 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:47:04.102150 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222914c9-8cd5-42f7-9035-455e293dfb4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:04.103143 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222914c9-8cd5-42f7-9035-455e293dfb4f None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:04.103565 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222914c9-8cd5-42f7-9035-455e293dfb4f None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:04.103982 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222914c9-8cd5-42f7-9035-455e293dfb4f None admin] PATH_INFO: `/v3/roles/service` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:04.116536 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222914c9-8cd5-42f7-9035-455e293dfb4f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 15:47:04.123176 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-222914c9-8cd5-42f7-9035-455e293dfb4f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 15:47:04.123176 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:47:04.123176 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:47:04.123176 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:47:04.123176 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:47:04.123176 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:47:04.123176 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:47:04.123176 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:47:04.123176 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:47:04.123176 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:47:04.123176 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:47:04.123176 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:47:04.123176 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 15:47:04.123176 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 15:47:04.123176 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 15:47:04.123176 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 15:47:04.124202 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 15:47:04.124202 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 15:47:04.124202 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 15:47:04.124202 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 15:47:04.124202 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:47:04.124202 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:47:04.124202 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:47:04.124202 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:47:04.124202 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:47:04.124202 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:47:04.124202 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:47:04.124202 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Aug 30 15:47:04.124202 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:47:04.124202 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:47:04.124202 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:47:04.124202 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:47:04.125108 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:47:04.125108 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:47:04.125108 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:47:04.125108 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:47:04.125108 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 15:47:04.125108 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 15:47:04.125108 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 15:47:04.125108 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 15:47:04.125108 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 15:47:04.125108 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 15:47:04.125108 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 15:47:04.125108 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Aug 30 15:47:04.127502 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 397/805] 10.222.0.20 () {56 vars in 1102 bytes} [Tue Aug 30 15:47:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 15:47:04.129108 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222914c9-8cd5-42f7-9035-455e293dfb4f None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:04.189312 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d7da1e-b254-4ae7-9670-21fe71221e89 None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:04.225083 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c649746a-42a9-4a06-be69-626d542938aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:04.225821 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c649746a-42a9-4a06-be69-626d542938aa None admin] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:04.226287 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c649746a-42a9-4a06-be69-626d542938aa None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:04.226667 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c649746a-42a9-4a06-be69-626d542938aa None admin] PATH_INFO: `/v3/endpoints` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:04.227533 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c649746a-42a9-4a06-be69-626d542938aa None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:04.228980 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c649746a-42a9-4a06-be69-626d542938aa None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:04.244440 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222914c9-8cd5-42f7-9035-455e293dfb4f None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:04.261244 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 409/806] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Aug 30 15:47:04 2022] POST /identity/v3/endpoints => generated 346 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 15:47:04.261244 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c649746a-42a9-4a06-be69-626d542938aa None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:04.318178 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e941bf8-35d0-4d51-a8d1-086f377bf55a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:04.318820 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e941bf8-35d0-4d51-a8d1-086f377bf55a None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:04.319099 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e941bf8-35d0-4d51-a8d1-086f377bf55a None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:04.319368 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e941bf8-35d0-4d51-a8d1-086f377bf55a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:04.323671 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:47:04.347738 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e941bf8-35d0-4d51-a8d1-086f377bf55a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:04.349276 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e941bf8-35d0-4d51-a8d1-086f377bf55a None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:04.377857 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c649746a-42a9-4a06-be69-626d542938aa None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:04.382029 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-4e941bf8-35d0-4d51-a8d1-086f377bf55a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 15:47:04.382029 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:47:04.382029 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:47:04.382029 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:47:04.382029 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:47:04.382029 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:47:04.382029 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:47:04.382029 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:47:04.382029 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:47:04.382029 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:47:04.382029 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:47:04.382029 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:47:04.382029 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 15:47:04.382029 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 15:47:04.382029 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 15:47:04.382029 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 15:47:04.383015 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:47:04.383015 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:47:04.383015 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:47:04.383015 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:47:04.383015 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:47:04.383015 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:47:04.383015 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:47:04.383015 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Aug 30 15:47:04.383015 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:47:04.383015 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:47:04.383015 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:47:04.383015 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:47:04.383015 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:47:04.383015 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:47:04.383015 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:47:04.383015 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:47:04.383015 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 15:47:04.383939 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 15:47:04.383939 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 15:47:04.383939 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Aug 30 15:47:04.383939 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 398/807] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 30 15:47:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 15:47:04.394476 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e941bf8-35d0-4d51-a8d1-086f377bf55a None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:04.421625 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b417d5-1e41-47e4-ab16-d5e07585e6ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:04.422258 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b417d5-1e41-47e4-ab16-d5e07585e6ef None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:04.422536 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b417d5-1e41-47e4-ab16-d5e07585e6ef None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:04.422812 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b417d5-1e41-47e4-ab16-d5e07585e6ef None admin] PATH_INFO: `/v3/roles` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:04.423543 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b417d5-1e41-47e4-ab16-d5e07585e6ef None admin] RBAC: Adding query filter params (name=service) {{(pid=81512) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:47:04.423846 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b417d5-1e41-47e4-ab16-d5e07585e6ef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:04.425758 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b417d5-1e41-47e4-ab16-d5e07585e6ef None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:04.445797 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 410/808] 10.222.0.20 () {56 vars in 1103 bytes} [Tue Aug 30 15:47:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:47:04.451152 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b417d5-1e41-47e4-ab16-d5e07585e6ef None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:04.469649 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:47:04.573602 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e941bf8-35d0-4d51-a8d1-086f377bf55a None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:04.581665 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b417d5-1e41-47e4-ab16-d5e07585e6ef None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:04.600409 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:47:04.623639 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3094ee2f-c47a-4f30-adbb-446ed824a0d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:04.623843 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8765fa9-6f56-4a31-af6b-4ff733274f68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:04.624269 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8765fa9-6f56-4a31-af6b-4ff733274f68 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:04.624380 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3094ee2f-c47a-4f30-adbb-446ed824a0d3 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:04.624538 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3094ee2f-c47a-4f30-adbb-446ed824a0d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:04.624629 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8765fa9-6f56-4a31-af6b-4ff733274f68 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:04.624806 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3094ee2f-c47a-4f30-adbb-446ed824a0d3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:04.624901 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8765fa9-6f56-4a31-af6b-4ff733274f68 None admin] PATH_INFO: `/v3/domains` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:04.626059 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8765fa9-6f56-4a31-af6b-4ff733274f68 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:47:04.626369 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8765fa9-6f56-4a31-af6b-4ff733274f68 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:04.627779 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8765fa9-6f56-4a31-af6b-4ff733274f68 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:04.647189 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3094ee2f-c47a-4f30-adbb-446ed824a0d3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:04.648495 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 399/809] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Aug 30 15:47:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:47:04.649043 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3094ee2f-c47a-4f30-adbb-446ed824a0d3 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:04.653846 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8765fa9-6f56-4a31-af6b-4ff733274f68 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:04.666882 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application [None req-3094ee2f-c47a-4f30-adbb-446ed824a0d3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 15:47:04.666882 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:47:04.666882 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:47:04.666882 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:47:04.666882 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:47:04.666882 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:47:04.666882 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:47:04.666882 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:47:04.666882 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:47:04.666882 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:47:04.666882 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:47:04.666882 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:47:04.666882 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 15:47:04.666882 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 15:47:04.666882 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 15:47:04.666882 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 15:47:04.667950 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:47:04.667950 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:47:04.667950 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:47:04.667950 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:47:04.667950 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:47:04.667950 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:47:04.667950 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:47:04.667950 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application with Lock( Aug 30 15:47:04.667950 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:47:04.667950 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:47:04.667950 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:47:04.667950 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:47:04.667950 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:47:04.667950 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:47:04.667950 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:47:04.667950 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:47:04.667950 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 15:47:04.668875 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 15:47:04.668875 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 15:47:04.668875 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Aug 30 15:47:04.668875 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 411/810] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 30 15:47:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 15:47:04.675136 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3094ee2f-c47a-4f30-adbb-446ed824a0d3 None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:04.800470 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3094ee2f-c47a-4f30-adbb-446ed824a0d3 None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:04.801387 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8765fa9-6f56-4a31-af6b-4ff733274f68 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:04.805535 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:47:04.844392 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d693c715-b83b-4b13-9620-b8a725288e9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:04.845501 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d693c715-b83b-4b13-9620-b8a725288e9e None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:04.846166 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d693c715-b83b-4b13-9620-b8a725288e9e None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:04.847021 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d693c715-b83b-4b13-9620-b8a725288e9e None admin] PATH_INFO: `/v3/users/nova` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:04.847380 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2535535e-024b-4433-9e90-24368d89b475 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:04.848310 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2535535e-024b-4433-9e90-24368d89b475 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:04.849019 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2535535e-024b-4433-9e90-24368d89b475 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:04.849506 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2535535e-024b-4433-9e90-24368d89b475 None admin] PATH_INFO: `/v3/domains` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:04.850969 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2535535e-024b-4433-9e90-24368d89b475 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81512) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:47:04.851462 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2535535e-024b-4433-9e90-24368d89b475 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:04.853044 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2535535e-024b-4433-9e90-24368d89b475 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:04.854640 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d693c715-b83b-4b13-9620-b8a725288e9e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:04.856770 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d693c715-b83b-4b13-9620-b8a725288e9e None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:04.868167 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 412/811] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Aug 30 15:47:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:47:04.869888 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de548ae5-cf0d-40f9-bee2-ea6e3c29321d None None] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:04.870715 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-d693c715-b83b-4b13-9620-b8a725288e9e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 30 15:47:04.870715 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:47:04.870715 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:47:04.870715 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:47:04.870715 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:47:04.870715 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:47:04.870715 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:47:04.870715 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:47:04.870715 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:47:04.870715 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:47:04.870715 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:47:04.870715 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:47:04.870715 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 15:47:04.870715 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 15:47:04.870715 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 15:47:04.870715 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 15:47:04.871666 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:47:04.871666 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:47:04.871666 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 15:47:04.871666 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 15:47:04.871666 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 15:47:04.871666 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 15:47:04.871666 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:47:04.871666 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:47:04.871666 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:47:04.871666 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:47:04.871666 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:47:04.871666 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Aug 30 15:47:04.871666 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:47:04.871666 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:47:04.871666 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:47:04.871666 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:47:04.871666 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:47:04.871666 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:47:04.872614 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:47:04.872614 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:47:04.872614 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 15:47:04.872614 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 15:47:04.872614 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 15:47:04.872614 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 15:47:04.872614 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 15:47:04.872614 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 15:47:04.872614 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 30 15:47:04.872614 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Aug 30 15:47:04.872614 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 400/812] 10.222.0.20 () {56 vars in 1093 bytes} [Tue Aug 30 15:47:04 2022] GET /identity/v3/users/nova => generated 82 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 15:47:04.873824 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d693c715-b83b-4b13-9620-b8a725288e9e None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:04.874191 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de548ae5-cf0d-40f9-bee2-ea6e3c29321d None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:04.874694 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de548ae5-cf0d-40f9-bee2-ea6e3c29321d None None] PATH_INFO: `/` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:04.876803 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 413/813] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 30 15:47:04 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 15:47:04.881742 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de548ae5-cf0d-40f9-bee2-ea6e3c29321d None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:04.918802 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:47:05.008235 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d693c715-b83b-4b13-9620-b8a725288e9e None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:05.015091 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de548ae5-cf0d-40f9-bee2-ea6e3c29321d None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:05.047893 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a02193-8683-4165-82db-c26cb9be4223 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:05.048538 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a02193-8683-4165-82db-c26cb9be4223 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:05.048820 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a02193-8683-4165-82db-c26cb9be4223 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:05.049151 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a02193-8683-4165-82db-c26cb9be4223 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:05.050964 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eebe966-612a-4873-be5d-525c14ef2492 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:05.051695 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eebe966-612a-4873-be5d-525c14ef2492 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:05.052063 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eebe966-612a-4873-be5d-525c14ef2492 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:05.052426 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eebe966-612a-4873-be5d-525c14ef2492 None admin] PATH_INFO: `/v3/users` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:05.053285 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eebe966-612a-4873-be5d-525c14ef2492 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81512) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:47:05.053688 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eebe966-612a-4873-be5d-525c14ef2492 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:05.055220 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eebe966-612a-4873-be5d-525c14ef2492 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:05.057309 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a02193-8683-4165-82db-c26cb9be4223 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:05.058896 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a02193-8683-4165-82db-c26cb9be4223 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:05.070782 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-14a02193-8683-4165-82db-c26cb9be4223 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 30 15:47:05.070782 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:47:05.070782 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:47:05.070782 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:47:05.070782 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:47:05.070782 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:47:05.070782 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:47:05.070782 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:47:05.070782 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:47:05.070782 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:47:05.070782 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:47:05.070782 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:47:05.070782 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 15:47:05.070782 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 15:47:05.070782 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 15:47:05.070782 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 15:47:05.071738 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:47:05.071738 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:47:05.071738 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 15:47:05.071738 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 15:47:05.071738 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 15:47:05.071738 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 15:47:05.071738 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:47:05.071738 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:47:05.071738 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:47:05.071738 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:47:05.071738 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:47:05.071738 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Aug 30 15:47:05.071738 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:47:05.071738 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:47:05.071738 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:47:05.071738 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:47:05.071738 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:47:05.071738 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:47:05.072695 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:47:05.072695 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:47:05.072695 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 15:47:05.072695 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 15:47:05.072695 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 15:47:05.072695 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 15:47:05.072695 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 15:47:05.072695 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 15:47:05.072695 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 30 15:47:05.072695 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Aug 30 15:47:05.072695 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 401/814] 10.222.0.20 () {56 vars in 1117 bytes} [Tue Aug 30 15:47:04 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 15:47:05.073607 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605bdf6c-5e6c-4880-b1f4-3c03d9fc2471 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:05.073721 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605bdf6c-5e6c-4880-b1f4-3c03d9fc2471 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:05.076567 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605bdf6c-5e6c-4880-b1f4-3c03d9fc2471 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:05.077632 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 414/815] 10.222.0.20 () {56 vars in 1133 bytes} [Tue Aug 30 15:47:04 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 15:47:05.078800 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eebe966-612a-4873-be5d-525c14ef2492 None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:05.092405 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:47:05.098841 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-605bdf6c-5e6c-4880-b1f4-3c03d9fc2471 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:47:05.099985 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-605bdf6c-5e6c-4880-b1f4-3c03d9fc2471 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:47:05.181525 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 402/816] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 30 15:47:05 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 15:47:05.189241 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:47:05.189394 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605bdf6c-5e6c-4880-b1f4-3c03d9fc2471 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:05.206439 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eebe966-612a-4873-be5d-525c14ef2492 None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:05.279156 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8cec27-26eb-4ed3-a077-47c5d524c9e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:05.279921 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8cec27-26eb-4ed3-a077-47c5d524c9e4 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:05.280159 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8cec27-26eb-4ed3-a077-47c5d524c9e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:05.280339 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8cec27-26eb-4ed3-a077-47c5d524c9e4 None admin] PATH_INFO: `/v3/users` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:05.281075 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8cec27-26eb-4ed3-a077-47c5d524c9e4 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81512) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:47:05.282395 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8cec27-26eb-4ed3-a077-47c5d524c9e4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:05.283886 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8cec27-26eb-4ed3-a077-47c5d524c9e4 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:05.321346 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 415/817] 10.222.0.20 () {56 vars in 1149 bytes} [Tue Aug 30 15:47:05 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:47:05.327070 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8cec27-26eb-4ed3-a077-47c5d524c9e4 None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:05.354003 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605bdf6c-5e6c-4880-b1f4-3c03d9fc2471 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:05.421788 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd054b0-e003-47ef-bc89-b67dae11f6f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:05.422411 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd054b0-e003-47ef-bc89-b67dae11f6f8 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:05.422684 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd054b0-e003-47ef-bc89-b67dae11f6f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:05.422952 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd054b0-e003-47ef-bc89-b67dae11f6f8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:05.429523 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:47:05.476814 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8cec27-26eb-4ed3-a077-47c5d524c9e4 None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:05.524902 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4b29b5-35fb-483f-b128-641ccf9c3787 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:05.525777 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4b29b5-35fb-483f-b128-641ccf9c3787 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:05.526053 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4b29b5-35fb-483f-b128-641ccf9c3787 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:05.526324 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4b29b5-35fb-483f-b128-641ccf9c3787 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:05.541202 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4b29b5-35fb-483f-b128-641ccf9c3787 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:05.542705 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4b29b5-35fb-483f-b128-641ccf9c3787 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:05.545939 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:47:05.559227 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application [None req-6e4b29b5-35fb-483f-b128-641ccf9c3787 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 15:47:05.559227 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:47:05.559227 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:47:05.559227 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:47:05.559227 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:47:05.559227 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:47:05.559227 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:47:05.559227 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:47:05.559227 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:47:05.559227 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:47:05.559227 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:47:05.559227 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:47:05.559227 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 15:47:05.559227 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 15:47:05.559227 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 15:47:05.559227 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 15:47:05.560243 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:47:05.560243 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:47:05.560243 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:47:05.560243 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:47:05.560243 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:47:05.560243 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:47:05.560243 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:47:05.560243 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application with Lock( Aug 30 15:47:05.560243 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:47:05.560243 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:47:05.560243 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:47:05.560243 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:47:05.560243 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:47:05.560243 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:47:05.560243 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:47:05.560243 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:47:05.560243 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 15:47:05.561750 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 15:47:05.561750 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 15:47:05.561750 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Aug 30 15:47:05.561750 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 416/818] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 30 15:47:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 15:47:05.561750 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4b29b5-35fb-483f-b128-641ccf9c3787 None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:05.568517 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd054b0-e003-47ef-bc89-b67dae11f6f8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:05.574123 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd054b0-e003-47ef-bc89-b67dae11f6f8 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:05.685020 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4b29b5-35fb-483f-b128-641ccf9c3787 None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:05.721140 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 403/819] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Aug 30 15:47:05 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 535 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 30 15:47:05.721140 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd054b0-e003-47ef-bc89-b67dae11f6f8 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:05.726088 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7546e85c-15ae-4916-aec1-e72c3e2f1127 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:05.727039 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7546e85c-15ae-4916-aec1-e72c3e2f1127 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:05.727518 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7546e85c-15ae-4916-aec1-e72c3e2f1127 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:05.728004 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7546e85c-15ae-4916-aec1-e72c3e2f1127 None admin] PATH_INFO: `/v3/services/network` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:05.728788 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7546e85c-15ae-4916-aec1-e72c3e2f1127 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:05.730505 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7546e85c-15ae-4916-aec1-e72c3e2f1127 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:05.740335 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application [None req-7546e85c-15ae-4916-aec1-e72c3e2f1127 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 30 15:47:05.740335 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:47:05.740335 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 15:47:05.740335 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 15:47:05.740335 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 15:47:05.740335 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 15:47:05.740335 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 15:47:05.740335 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 15:47:05.740335 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 30 15:47:05.740335 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Aug 30 15:47:05.740335 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 15:47:05.740335 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Aug 30 15:47:05.740335 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:47:05.740335 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:47:05.740335 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:47:05.740335 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:47:05.740335 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:47:05.741423 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:47:05.741423 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:47:05.741423 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:47:05.741423 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:47:05.741423 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:47:05.741423 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:47:05.741423 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 15:47:05.741423 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 15:47:05.741423 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 15:47:05.741423 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 15:47:05.741423 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:47:05.741423 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:47:05.741423 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:47:05.741423 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:47:05.741423 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:47:05.741423 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:47:05.742312 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:47:05.742312 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application with Lock( Aug 30 15:47:05.742312 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:47:05.742312 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:47:05.742312 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:47:05.742312 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:47:05.742312 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:47:05.742312 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:47:05.742312 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:47:05.742312 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:47:05.742312 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 15:47:05.742312 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 15:47:05.742312 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 30 15:47:05.742312 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Aug 30 15:47:05.744894 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 417/820] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Aug 30 15:47:05 2022] GET /identity/v3/services/network => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 15:47:05.746254 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7546e85c-15ae-4916-aec1-e72c3e2f1127 None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:05.756233 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:47:05.808590 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:47:05.826251 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd054b0-e003-47ef-bc89-b67dae11f6f8 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:05.853896 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7546e85c-15ae-4916-aec1-e72c3e2f1127 None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:05.865824 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55bedb9-b063-4545-97ca-d7fef392a15f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:05.866602 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55bedb9-b063-4545-97ca-d7fef392a15f None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:05.866882 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55bedb9-b063-4545-97ca-d7fef392a15f None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:05.867154 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55bedb9-b063-4545-97ca-d7fef392a15f None admin] PATH_INFO: `/v3/domains` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:05.867808 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55bedb9-b063-4545-97ca-d7fef392a15f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:47:05.868111 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55bedb9-b063-4545-97ca-d7fef392a15f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:05.869521 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55bedb9-b063-4545-97ca-d7fef392a15f None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:05.881793 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 404/821] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Aug 30 15:47:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:47:05.881793 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0e1dd4-b104-4e65-9c86-a5c6be5efe10 None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:05.881793 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0e1dd4-b104-4e65-9c86-a5c6be5efe10 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:05.881793 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0e1dd4-b104-4e65-9c86-a5c6be5efe10 None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:05.881793 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 405/822] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 30 15:47:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 15:47:05.881793 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0e1dd4-b104-4e65-9c86-a5c6be5efe10 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:05.900081 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ba3ff0-7698-4547-bb57-823232b3fd1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:05.900672 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ba3ff0-7698-4547-bb57-823232b3fd1e None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:05.900953 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ba3ff0-7698-4547-bb57-823232b3fd1e None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:05.901210 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ba3ff0-7698-4547-bb57-823232b3fd1e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:05.913624 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ba3ff0-7698-4547-bb57-823232b3fd1e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:05.915164 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ba3ff0-7698-4547-bb57-823232b3fd1e None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:05.930166 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application [None req-d2ba3ff0-7698-4547-bb57-823232b3fd1e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 15:47:05.930166 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:47:05.930166 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:47:05.930166 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:47:05.930166 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:47:05.930166 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:47:05.930166 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:47:05.930166 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:47:05.930166 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:47:05.930166 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:47:05.930166 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:47:05.930166 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:47:05.930166 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 15:47:05.930166 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 15:47:05.930166 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 15:47:05.930166 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 15:47:05.931185 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:47:05.931185 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:47:05.931185 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:47:05.931185 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:47:05.931185 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:47:05.931185 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:47:05.931185 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:47:05.931185 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application with Lock( Aug 30 15:47:05.931185 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:47:05.931185 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:47:05.931185 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:47:05.931185 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:47:05.931185 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:47:05.931185 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:47:05.931185 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:47:05.931185 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:47:05.931185 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 15:47:05.932081 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 15:47:05.932081 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 15:47:05.932081 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Aug 30 15:47:05.932081 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 418/823] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 30 15:47:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 15:47:05.933162 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a3ed36-5059-4be4-a64c-feb025f2c851 None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:05.933162 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a3ed36-5059-4be4-a64c-feb025f2c851 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:05.933162 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a3ed36-5059-4be4-a64c-feb025f2c851 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:05.952534 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-84a3ed36-5059-4be4-a64c-feb025f2c851 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:47:05.953920 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-84a3ed36-5059-4be4-a64c-feb025f2c851 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:47:05.996304 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:47:06.004881 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0e1dd4-b104-4e65-9c86-a5c6be5efe10 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:06.029282 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 419/824] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 30 15:47:05 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 15:47:06.029282 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a3ed36-5059-4be4-a64c-feb025f2c851 None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:06.044715 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c4c9e8-6320-4d28-b450-fb76d2dc9831 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:06.045498 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c4c9e8-6320-4d28-b450-fb76d2dc9831 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:06.045965 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c4c9e8-6320-4d28-b450-fb76d2dc9831 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:06.046309 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c4c9e8-6320-4d28-b450-fb76d2dc9831 None admin] PATH_INFO: `/v3/services` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:06.047023 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c4c9e8-6320-4d28-b450-fb76d2dc9831 None admin] RBAC: Adding query filter params (name=network) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:47:06.047397 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c4c9e8-6320-4d28-b450-fb76d2dc9831 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:06.049024 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c4c9e8-6320-4d28-b450-fb76d2dc9831 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:06.058592 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 406/825] 10.222.0.20 () {56 vars in 1112 bytes} [Tue Aug 30 15:47:05 2022] GET /identity/v3/services?name=network => generated 124 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:47:06.058592 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c4c9e8-6320-4d28-b450-fb76d2dc9831 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:06.086283 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:47:06.145133 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a3ed36-5059-4be4-a64c-feb025f2c851 None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:06.165089 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c4c9e8-6320-4d28-b450-fb76d2dc9831 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:06.180112 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de03d78-06e1-4092-ab2f-3d3d1fe8652e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:06.180570 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de03d78-06e1-4092-ab2f-3d3d1fe8652e None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:06.180847 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de03d78-06e1-4092-ab2f-3d3d1fe8652e None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:06.181114 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de03d78-06e1-4092-ab2f-3d3d1fe8652e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:06.188085 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de03d78-06e1-4092-ab2f-3d3d1fe8652e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:06.189565 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de03d78-06e1-4092-ab2f-3d3d1fe8652e None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:06.201536 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application [None req-7de03d78-06e1-4092-ab2f-3d3d1fe8652e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 15:47:06.201536 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:47:06.201536 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:47:06.201536 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:47:06.201536 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:47:06.201536 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:47:06.201536 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:47:06.201536 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:47:06.201536 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:47:06.201536 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:47:06.201536 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:47:06.201536 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:47:06.201536 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 15:47:06.201536 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 15:47:06.201536 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 15:47:06.202493 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 15:47:06.202493 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:47:06.202493 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:47:06.202493 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:47:06.202493 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:47:06.202493 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:47:06.202493 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:47:06.202493 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:47:06.202493 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application with Lock( Aug 30 15:47:06.202493 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:47:06.202493 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:47:06.202493 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:47:06.202493 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:47:06.202493 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:47:06.202493 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:47:06.202493 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:47:06.202493 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:47:06.203391 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 15:47:06.203391 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 15:47:06.203391 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 15:47:06.203391 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 15:47:06.203391 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 15:47:06.203391 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 15:47:06.203391 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 15:47:06.203391 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Aug 30 15:47:06.205357 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 420/826] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Aug 30 15:47:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 15:47:06.206364 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de03d78-06e1-4092-ab2f-3d3d1fe8652e None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:06.208044 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6454e9-6df4-40e5-9451-4cd81d4374b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:06.208914 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6454e9-6df4-40e5-9451-4cd81d4374b4 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:06.209829 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6454e9-6df4-40e5-9451-4cd81d4374b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:06.210294 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6454e9-6df4-40e5-9451-4cd81d4374b4 None admin] PATH_INFO: `/v3/domains` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:06.213049 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6454e9-6df4-40e5-9451-4cd81d4374b4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:47:06.213501 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6454e9-6df4-40e5-9451-4cd81d4374b4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:06.214953 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6454e9-6df4-40e5-9451-4cd81d4374b4 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:06.233500 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 407/827] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Aug 30 15:47:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:47:06.233500 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6454e9-6df4-40e5-9451-4cd81d4374b4 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:06.257516 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:47:06.329448 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de03d78-06e1-4092-ab2f-3d3d1fe8652e None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:06.331374 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:47:06.368066 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5627f945-4cd9-4bb2-aa28-ec0ba98324dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:06.368692 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5627f945-4cd9-4bb2-aa28-ec0ba98324dd None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:06.368972 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5627f945-4cd9-4bb2-aa28-ec0ba98324dd None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:06.369292 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5627f945-4cd9-4bb2-aa28-ec0ba98324dd None admin] PATH_INFO: `/v3/services` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:06.369914 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5627f945-4cd9-4bb2-aa28-ec0ba98324dd None admin] RBAC: Adding query filter params (type=network) {{(pid=81512) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:47:06.370216 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5627f945-4cd9-4bb2-aa28-ec0ba98324dd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:06.371959 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5627f945-4cd9-4bb2-aa28-ec0ba98324dd None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:06.382932 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 421/828] 10.222.0.20 () {56 vars in 1112 bytes} [Tue Aug 30 15:47:06 2022] GET /identity/v3/services?type=network => generated 350 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:47:06.382932 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5627f945-4cd9-4bb2-aa28-ec0ba98324dd None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:06.465853 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6454e9-6df4-40e5-9451-4cd81d4374b4 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:06.492396 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5627f945-4cd9-4bb2-aa28-ec0ba98324dd None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:06.499671 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1d1714-ea12-4fc7-813c-198bee24b7cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:06.500692 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1d1714-ea12-4fc7-813c-198bee24b7cb None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:06.500971 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1d1714-ea12-4fc7-813c-198bee24b7cb None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:06.501262 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1d1714-ea12-4fc7-813c-198bee24b7cb None admin] PATH_INFO: `/v3/projects` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:06.501969 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1d1714-ea12-4fc7-813c-198bee24b7cb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:47:06.502271 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1d1714-ea12-4fc7-813c-198bee24b7cb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:06.503721 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1d1714-ea12-4fc7-813c-198bee24b7cb None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:06.516026 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 408/829] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Aug 30 15:47:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:47:06.516912 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1d1714-ea12-4fc7-813c-198bee24b7cb None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:06.527101 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e045746e-5c47-4d28-931c-297653d12939 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:06.527736 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e045746e-5c47-4d28-931c-297653d12939 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:06.528007 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e045746e-5c47-4d28-931c-297653d12939 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:06.528274 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e045746e-5c47-4d28-931c-297653d12939 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:06.535292 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e045746e-5c47-4d28-931c-297653d12939 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:06.536746 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e045746e-5c47-4d28-931c-297653d12939 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:06.550463 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application [None req-e045746e-5c47-4d28-931c-297653d12939 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 15:47:06.550463 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:47:06.550463 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:47:06.550463 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:47:06.550463 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:47:06.550463 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:47:06.550463 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:47:06.550463 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:47:06.550463 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:47:06.550463 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:47:06.550463 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:47:06.550463 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:47:06.550463 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 15:47:06.550463 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 15:47:06.550463 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 15:47:06.551520 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 15:47:06.551520 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:47:06.551520 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:47:06.551520 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:47:06.551520 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:47:06.551520 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:47:06.551520 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:47:06.551520 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:47:06.551520 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application with Lock( Aug 30 15:47:06.551520 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:47:06.551520 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:47:06.551520 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:47:06.551520 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:47:06.551520 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:47:06.551520 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:47:06.551520 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:47:06.551520 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:47:06.552602 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 15:47:06.552602 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 15:47:06.552602 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 15:47:06.552602 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 15:47:06.552602 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 15:47:06.552602 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 15:47:06.552602 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 15:47:06.552602 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Aug 30 15:47:06.552602 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 422/830] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Aug 30 15:47:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 15:47:06.553206 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e045746e-5c47-4d28-931c-297653d12939 None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:06.579820 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:47:06.602712 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:47:06.623180 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1d1714-ea12-4fc7-813c-198bee24b7cb None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:06.655946 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e045746e-5c47-4d28-931c-297653d12939 None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:06.660597 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b67f39-dbde-4b9b-ab5f-3b80b4992771 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:06.661236 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b67f39-dbde-4b9b-ab5f-3b80b4992771 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:06.661525 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b67f39-dbde-4b9b-ab5f-3b80b4992771 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:06.661792 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b67f39-dbde-4b9b-ab5f-3b80b4992771 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:06.671879 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b67f39-dbde-4b9b-ab5f-3b80b4992771 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 15:47:06.676690 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-71b67f39-dbde-4b9b-ab5f-3b80b4992771 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 15:47:06.676690 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:47:06.676690 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:47:06.676690 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:47:06.676690 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:47:06.676690 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:47:06.676690 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:47:06.676690 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:47:06.676690 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:47:06.676690 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:47:06.676690 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:47:06.676690 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:47:06.676690 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 15:47:06.676690 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 15:47:06.676690 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 15:47:06.676690 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 15:47:06.677899 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 15:47:06.677899 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 15:47:06.677899 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 15:47:06.677899 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 15:47:06.677899 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:47:06.677899 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:47:06.677899 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:47:06.677899 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:47:06.677899 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:47:06.677899 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:47:06.677899 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:47:06.677899 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Aug 30 15:47:06.677899 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:47:06.677899 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:47:06.677899 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:47:06.677899 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:47:06.678742 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:47:06.678742 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:47:06.678742 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:47:06.678742 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:47:06.678742 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 15:47:06.678742 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 15:47:06.678742 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 15:47:06.678742 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 15:47:06.678742 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 15:47:06.678742 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 15:47:06.678742 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 15:47:06.678742 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Aug 30 15:47:06.678742 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 409/831] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Aug 30 15:47:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 15:47:06.679553 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517962d6-3ffc-4ce3-b5ef-9ef5122e92fc None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:06.679832 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517962d6-3ffc-4ce3-b5ef-9ef5122e92fc None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:06.680101 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517962d6-3ffc-4ce3-b5ef-9ef5122e92fc None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:06.685153 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 410/832] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 30 15:47:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 15:47:06.685153 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517962d6-3ffc-4ce3-b5ef-9ef5122e92fc None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:06.691869 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e567508-e736-433c-8b1b-20c4e71e3cfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:06.692475 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e567508-e736-433c-8b1b-20c4e71e3cfe None admin] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:06.692746 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e567508-e736-433c-8b1b-20c4e71e3cfe None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:06.693012 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e567508-e736-433c-8b1b-20c4e71e3cfe None admin] PATH_INFO: `/v3/endpoints` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:06.693724 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e567508-e736-433c-8b1b-20c4e71e3cfe None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:06.695088 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e567508-e736-433c-8b1b-20c4e71e3cfe None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:06.716870 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 423/833] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Aug 30 15:47:06 2022] POST /identity/v3/endpoints => generated 337 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 15:47:06.717706 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e567508-e736-433c-8b1b-20c4e71e3cfe None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:06.800144 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517962d6-3ffc-4ce3-b5ef-9ef5122e92fc None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:06.819047 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:47:06.839167 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a356c0f0-b957-4ec7-83b6-e86df2ff9b8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:06.839838 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a356c0f0-b957-4ec7-83b6-e86df2ff9b8a None admin] REQUEST_METHOD: `PUT` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:06.840107 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a356c0f0-b957-4ec7-83b6-e86df2ff9b8a None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:06.840374 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a356c0f0-b957-4ec7-83b6-e86df2ff9b8a None admin] PATH_INFO: `/v3/projects/854fe71b49224dff9a4b1766a20ab266/users/07130d598131443486c2bc1c6673c45b/roles/57eb8466096346b39319536c6feeb6f7` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:06.842057 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e567508-e736-433c-8b1b-20c4e71e3cfe None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:06.845573 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:47:06.859223 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a356c0f0-b957-4ec7-83b6-e86df2ff9b8a None admin] RBAC: Authorizing `identity:create_grant(project_id=854fe71b49224dff9a4b1766a20ab266, user_id=07130d598131443486c2bc1c6673c45b, role_id=57eb8466096346b39319536c6feeb6f7)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:06.860831 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a356c0f0-b957-4ec7-83b6-e86df2ff9b8a None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:06.876941 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 411/834] 10.222.0.20 () {58 vars in 1439 bytes} [Tue Aug 30 15:47:06 2022] PUT /identity/v3/projects/854fe71b49224dff9a4b1766a20ab266/users/07130d598131443486c2bc1c6673c45b/roles/57eb8466096346b39319536c6feeb6f7 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:47:06.877965 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a356c0f0-b957-4ec7-83b6-e86df2ff9b8a None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:06.883098 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d951dca4-723b-40e6-9ac6-6cbdb66d9d40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:06.883729 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d951dca4-723b-40e6-9ac6-6cbdb66d9d40 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:06.884003 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d951dca4-723b-40e6-9ac6-6cbdb66d9d40 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:06.884270 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d951dca4-723b-40e6-9ac6-6cbdb66d9d40 None admin] PATH_INFO: `/v3/projects` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:06.884973 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d951dca4-723b-40e6-9ac6-6cbdb66d9d40 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81512) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:47:06.885320 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d951dca4-723b-40e6-9ac6-6cbdb66d9d40 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:06.887371 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d951dca4-723b-40e6-9ac6-6cbdb66d9d40 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:06.902230 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 424/835] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Aug 30 15:47:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:47:06.908536 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8f3cb6-b0ac-4c48-b617-adf5c79a3a9b None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:06.908536 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8f3cb6-b0ac-4c48-b617-adf5c79a3a9b None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:06.908536 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8f3cb6-b0ac-4c48-b617-adf5c79a3a9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:06.926108 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-0f8f3cb6-b0ac-4c48-b617-adf5c79a3a9b None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:47:06.927397 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-0f8f3cb6-b0ac-4c48-b617-adf5c79a3a9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:47:07.016388 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a356c0f0-b957-4ec7-83b6-e86df2ff9b8a None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:07.023779 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 425/836] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 30 15:47:06 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 15:47:07.024878 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8f3cb6-b0ac-4c48-b617-adf5c79a3a9b None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:07.062756 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41e8258-9463-476c-8d64-a8597f8d48f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:07.063378 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41e8258-9463-476c-8d64-a8597f8d48f1 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:07.063657 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41e8258-9463-476c-8d64-a8597f8d48f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:07.063928 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41e8258-9463-476c-8d64-a8597f8d48f1 None admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:07.064604 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41e8258-9463-476c-8d64-a8597f8d48f1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:47:07.064912 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41e8258-9463-476c-8d64-a8597f8d48f1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:07.066690 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41e8258-9463-476c-8d64-a8597f8d48f1 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:07.075094 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 412/837] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Aug 30 15:47:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:47:07.081065 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41e8258-9463-476c-8d64-a8597f8d48f1 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:07.114499 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:47:07.140898 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:47:07.155174 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8f3cb6-b0ac-4c48-b617-adf5c79a3a9b None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:07.197745 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1500a041-969e-4126-9ae9-92a0fdea0016 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:07.198357 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1500a041-969e-4126-9ae9-92a0fdea0016 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:07.198779 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1500a041-969e-4126-9ae9-92a0fdea0016 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:07.199048 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1500a041-969e-4126-9ae9-92a0fdea0016 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:07.200073 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1500a041-969e-4126-9ae9-92a0fdea0016 None admin] RBAC: Adding query filter params (role.id=f265386950a641ac8413b53eda4ae34d, scope.project.id=854fe71b49224dff9a4b1766a20ab266, user.id=7a1e756ae9be4a7bac1e81acf07e8f3b) {{(pid=81512) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:47:07.200661 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1500a041-969e-4126-9ae9-92a0fdea0016 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:07.202694 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1500a041-969e-4126-9ae9-92a0fdea0016 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:07.205387 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41e8258-9463-476c-8d64-a8597f8d48f1 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:07.217317 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 426/838] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Aug 30 15:47:07 2022] GET /identity/v3/role_assignments?user.id=7a1e756ae9be4a7bac1e81acf07e8f3b&scope.project.id=854fe71b49224dff9a4b1766a20ab266&role.id=f265386950a641ac8413b53eda4ae34d => generated 608 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:47:07.248094 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae502bfb-b474-4fd4-8550-5a97c7af8ff5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:07.248784 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae502bfb-b474-4fd4-8550-5a97c7af8ff5 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:07.249060 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae502bfb-b474-4fd4-8550-5a97c7af8ff5 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:07.249388 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae502bfb-b474-4fd4-8550-5a97c7af8ff5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:07.261530 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae502bfb-b474-4fd4-8550-5a97c7af8ff5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:07.263007 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae502bfb-b474-4fd4-8550-5a97c7af8ff5 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:07.279349 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-ae502bfb-b474-4fd4-8550-5a97c7af8ff5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 15:47:07.279349 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:47:07.279349 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:47:07.279349 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:47:07.279349 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:47:07.279349 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:47:07.279349 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:47:07.279349 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:47:07.279349 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:47:07.279349 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:47:07.279349 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:47:07.279349 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:47:07.279349 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 15:47:07.279349 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 15:47:07.279349 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 15:47:07.279349 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 15:47:07.280553 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:47:07.280553 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:47:07.280553 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:47:07.280553 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:47:07.280553 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:47:07.280553 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:47:07.280553 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:47:07.280553 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Aug 30 15:47:07.280553 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:47:07.280553 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:47:07.280553 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:47:07.280553 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:47:07.280553 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:47:07.280553 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:47:07.280553 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:47:07.280553 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:47:07.280553 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 15:47:07.282226 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 15:47:07.282226 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 15:47:07.282226 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Aug 30 15:47:07.282226 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 413/839] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 30 15:47:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 15:47:07.286966 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1500a041-969e-4126-9ae9-92a0fdea0016 None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:07.294170 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae502bfb-b474-4fd4-8550-5a97c7af8ff5 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:07.406261 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1500a041-969e-4126-9ae9-92a0fdea0016 None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:07.417173 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:47:07.420470 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae502bfb-b474-4fd4-8550-5a97c7af8ff5 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:07.454091 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe61d04-9f2c-413a-a344-1be134d7ba4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:07.455762 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe61d04-9f2c-413a-a344-1be134d7ba4a None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:07.455762 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe61d04-9f2c-413a-a344-1be134d7ba4a None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:07.455762 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe61d04-9f2c-413a-a344-1be134d7ba4a None admin] PATH_INFO: `/v3/domains` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:07.455762 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe61d04-9f2c-413a-a344-1be134d7ba4a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81512) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:47:07.456067 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe61d04-9f2c-413a-a344-1be134d7ba4a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:07.458090 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe61d04-9f2c-413a-a344-1be134d7ba4a None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:07.459965 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:47:07.468325 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3fa53c-28c8-4689-b4e0-7851a50dca40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:07.469057 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3fa53c-28c8-4689-b4e0-7851a50dca40 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:07.469366 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3fa53c-28c8-4689-b4e0-7851a50dca40 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:07.469637 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3fa53c-28c8-4689-b4e0-7851a50dca40 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:07.470332 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3fa53c-28c8-4689-b4e0-7851a50dca40 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:07.471103 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 427/840] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Aug 30 15:47:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:47:07.471867 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3fa53c-28c8-4689-b4e0-7851a50dca40 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:07.476298 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe61d04-9f2c-413a-a344-1be134d7ba4a None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:07.483241 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-4c3fa53c-28c8-4689-b4e0-7851a50dca40 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 30 15:47:07.483241 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:47:07.483241 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 15:47:07.483241 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 15:47:07.483241 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 15:47:07.483241 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 15:47:07.483241 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 15:47:07.483241 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 15:47:07.483241 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 30 15:47:07.483241 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Aug 30 15:47:07.483241 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 15:47:07.483241 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Aug 30 15:47:07.483241 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:47:07.483241 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:47:07.483241 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:47:07.483241 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:47:07.483241 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:47:07.484492 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:47:07.484492 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:47:07.484492 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:47:07.484492 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:47:07.484492 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:47:07.484492 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:47:07.484492 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 15:47:07.484492 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 15:47:07.484492 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 15:47:07.484492 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 15:47:07.484492 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:47:07.484492 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:47:07.484492 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:47:07.484492 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:47:07.484492 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:47:07.484492 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:47:07.485530 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:47:07.485530 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Aug 30 15:47:07.485530 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:47:07.485530 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:47:07.485530 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:47:07.485530 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:47:07.485530 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:47:07.485530 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:47:07.485530 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:47:07.485530 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:47:07.485530 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 15:47:07.485530 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 15:47:07.485530 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 30 15:47:07.485530 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Aug 30 15:47:07.485530 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 414/841] 10.222.0.20 () {56 vars in 1114 bytes} [Tue Aug 30 15:47:07 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 15:47:07.489988 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3fa53c-28c8-4689-b4e0-7851a50dca40 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:07.602980 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe61d04-9f2c-413a-a344-1be134d7ba4a None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:07.607908 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3fa53c-28c8-4689-b4e0-7851a50dca40 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:07.650976 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe1f3c2-9c29-4ef8-88d3-4d95f9ca9924 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:07.652001 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe1f3c2-9c29-4ef8-88d3-4d95f9ca9924 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:07.652479 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe1f3c2-9c29-4ef8-88d3-4d95f9ca9924 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:07.652953 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe1f3c2-9c29-4ef8-88d3-4d95f9ca9924 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:07.655462 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e11a32-e6ee-4270-a278-edf94bb93555 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:07.655997 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e11a32-e6ee-4270-a278-edf94bb93555 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:07.656273 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e11a32-e6ee-4270-a278-edf94bb93555 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:07.656540 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e11a32-e6ee-4270-a278-edf94bb93555 None admin] PATH_INFO: `/v3/services` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:07.657593 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e11a32-e6ee-4270-a278-edf94bb93555 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:47:07.657593 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e11a32-e6ee-4270-a278-edf94bb93555 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:07.658881 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e11a32-e6ee-4270-a278-edf94bb93555 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:07.662586 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe1f3c2-9c29-4ef8-88d3-4d95f9ca9924 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:07.664281 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 415/842] 10.222.0.20 () {56 vars in 1114 bytes} [Tue Aug 30 15:47:07 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 15:47:07.664458 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe1f3c2-9c29-4ef8-88d3-4d95f9ca9924 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:07.669390 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e11a32-e6ee-4270-a278-edf94bb93555 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:07.676994 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application [None req-6fe1f3c2-9c29-4ef8-88d3-4d95f9ca9924 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 30 15:47:07.676994 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:47:07.676994 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:47:07.676994 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:47:07.676994 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:47:07.676994 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:47:07.676994 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:47:07.676994 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:47:07.676994 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:47:07.676994 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:47:07.676994 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:47:07.676994 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:47:07.676994 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 15:47:07.676994 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 15:47:07.676994 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 15:47:07.676994 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 15:47:07.677983 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:47:07.677983 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:47:07.677983 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 15:47:07.677983 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 15:47:07.677983 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 15:47:07.677983 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 15:47:07.677983 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:47:07.677983 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:47:07.677983 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:47:07.677983 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:47:07.677983 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:47:07.677983 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application with Lock( Aug 30 15:47:07.677983 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:47:07.677983 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:47:07.677983 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:47:07.677983 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:47:07.677983 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:47:07.677983 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:47:07.679218 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:47:07.679218 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:47:07.679218 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 15:47:07.679218 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 15:47:07.679218 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 15:47:07.679218 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 15:47:07.679218 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 15:47:07.679218 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 15:47:07.679218 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 30 15:47:07.679218 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Aug 30 15:47:07.679218 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 428/843] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Aug 30 15:47:07 2022] GET /identity/v3/users/swift => generated 83 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 15:47:07.683651 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe1f3c2-9c29-4ef8-88d3-4d95f9ca9924 None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:07.705913 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:47:07.757658 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:47:07.786086 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e11a32-e6ee-4270-a278-edf94bb93555 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:07.818413 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe1f3c2-9c29-4ef8-88d3-4d95f9ca9924 None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:07.822332 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534304e8-0075-4eb8-a94b-a6e161485a1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:07.822946 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534304e8-0075-4eb8-a94b-a6e161485a1b None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:07.823219 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534304e8-0075-4eb8-a94b-a6e161485a1b None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:07.823485 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534304e8-0075-4eb8-a94b-a6e161485a1b None admin] PATH_INFO: `/v3/services` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:07.824119 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534304e8-0075-4eb8-a94b-a6e161485a1b None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:47:07.824416 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534304e8-0075-4eb8-a94b-a6e161485a1b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:07.826046 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534304e8-0075-4eb8-a94b-a6e161485a1b None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:07.836868 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 416/844] 10.222.0.20 () {56 vars in 1114 bytes} [Tue Aug 30 15:47:07 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:47:07.868856 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7216f45f-1528-4675-b54f-fee712b8fa50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:07.869469 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7216f45f-1528-4675-b54f-fee712b8fa50 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:07.869742 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7216f45f-1528-4675-b54f-fee712b8fa50 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:07.870057 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7216f45f-1528-4675-b54f-fee712b8fa50 None admin] PATH_INFO: `/v3/users` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:07.871213 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7216f45f-1528-4675-b54f-fee712b8fa50 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81512) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:47:07.871517 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7216f45f-1528-4675-b54f-fee712b8fa50 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:07.872992 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7216f45f-1528-4675-b54f-fee712b8fa50 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:07.893766 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 429/845] 10.222.0.20 () {56 vars in 1135 bytes} [Tue Aug 30 15:47:07 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:47:07.902456 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534304e8-0075-4eb8-a94b-a6e161485a1b None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:08.011180 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:47:08.021620 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534304e8-0075-4eb8-a94b-a6e161485a1b None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:08.057332 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b7d9b2-3510-451e-ba95-41261b3ba407 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:08.058251 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b7d9b2-3510-451e-ba95-41261b3ba407 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:08.058702 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b7d9b2-3510-451e-ba95-41261b3ba407 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:08.059140 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b7d9b2-3510-451e-ba95-41261b3ba407 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:08.131349 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b7d9b2-3510-451e-ba95-41261b3ba407 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:08.133088 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b7d9b2-3510-451e-ba95-41261b3ba407 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:08.223522 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:47:08.233263 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 417/846] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Aug 30 15:47:07 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 30 15:47:08.240166 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7216f45f-1528-4675-b54f-fee712b8fa50 None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:08.342048 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7216f45f-1528-4675-b54f-fee712b8fa50 None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:08.356560 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:47:08.377574 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e14b64-cf3e-4667-909d-8d05bd0f8a07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:08.378217 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e14b64-cf3e-4667-909d-8d05bd0f8a07 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:08.378494 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e14b64-cf3e-4667-909d-8d05bd0f8a07 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:08.378760 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e14b64-cf3e-4667-909d-8d05bd0f8a07 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:08.389350 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e14b64-cf3e-4667-909d-8d05bd0f8a07 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:08.390753 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e14b64-cf3e-4667-909d-8d05bd0f8a07 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:08.406035 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application [None req-84e14b64-cf3e-4667-909d-8d05bd0f8a07 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 15:47:08.406035 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:47:08.406035 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:47:08.406035 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:47:08.406035 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:47:08.406035 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:47:08.406035 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:47:08.406035 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:47:08.406035 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:47:08.406035 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:47:08.406035 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:47:08.406035 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:47:08.406035 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 15:47:08.406035 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 15:47:08.406035 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 15:47:08.406035 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 15:47:08.407532 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:47:08.407532 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:47:08.407532 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:47:08.407532 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:47:08.407532 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:47:08.407532 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:47:08.407532 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:47:08.407532 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application with Lock( Aug 30 15:47:08.407532 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:47:08.407532 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:47:08.407532 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:47:08.407532 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:47:08.407532 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:47:08.407532 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:47:08.407532 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:47:08.407532 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:47:08.407532 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 15:47:08.408444 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 15:47:08.408444 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 15:47:08.408444 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Aug 30 15:47:08.408444 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 430/847] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 30 15:47:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 15:47:08.415262 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b7d9b2-3510-451e-ba95-41261b3ba407 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:08.518893 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b7d9b2-3510-451e-ba95-41261b3ba407 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:08.554653 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210164c2-b953-48cd-9b56-4c56856ce125 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:08.555310 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210164c2-b953-48cd-9b56-4c56856ce125 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:08.555585 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210164c2-b953-48cd-9b56-4c56856ce125 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:08.555848 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210164c2-b953-48cd-9b56-4c56856ce125 None admin] PATH_INFO: `/v3/domains` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:08.556488 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210164c2-b953-48cd-9b56-4c56856ce125 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:47:08.556782 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210164c2-b953-48cd-9b56-4c56856ce125 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:08.558179 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210164c2-b953-48cd-9b56-4c56856ce125 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:08.570257 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 418/848] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Aug 30 15:47:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:47:08.575877 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e14b64-cf3e-4667-909d-8d05bd0f8a07 None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:08.691618 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e14b64-cf3e-4667-909d-8d05bd0f8a07 None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:08.726253 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d71522a-1444-4ac7-a553-dde361816c52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:08.726889 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d71522a-1444-4ac7-a553-dde361816c52 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:08.727167 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d71522a-1444-4ac7-a553-dde361816c52 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:08.727445 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d71522a-1444-4ac7-a553-dde361816c52 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:08.734583 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d71522a-1444-4ac7-a553-dde361816c52 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:08.736025 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d71522a-1444-4ac7-a553-dde361816c52 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:08.747903 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application [None req-3d71522a-1444-4ac7-a553-dde361816c52 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 15:47:08.747903 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:47:08.747903 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:47:08.747903 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:47:08.747903 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:47:08.747903 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:47:08.747903 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:47:08.747903 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:47:08.747903 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:47:08.747903 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:47:08.747903 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:47:08.747903 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:47:08.747903 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 15:47:08.747903 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 15:47:08.747903 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 15:47:08.748843 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 15:47:08.748843 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:47:08.748843 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:47:08.748843 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:47:08.748843 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:47:08.748843 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:47:08.748843 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:47:08.748843 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:47:08.748843 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application with Lock( Aug 30 15:47:08.748843 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:47:08.748843 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:47:08.748843 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:47:08.748843 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:47:08.748843 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:47:08.748843 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:47:08.748843 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:47:08.748843 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:47:08.749879 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 15:47:08.749879 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 15:47:08.749879 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 15:47:08.749879 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 15:47:08.749879 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 15:47:08.749879 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 15:47:08.749879 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 15:47:08.749879 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Aug 30 15:47:08.749879 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 431/849] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Aug 30 15:47:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 15:47:08.755274 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210164c2-b953-48cd-9b56-4c56856ce125 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:08.810015 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:47:08.859700 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210164c2-b953-48cd-9b56-4c56856ce125 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:08.894647 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd831ad-97d8-48d2-b68c-1ebbaabd1424 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:08.895174 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd831ad-97d8-48d2-b68c-1ebbaabd1424 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:08.895452 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd831ad-97d8-48d2-b68c-1ebbaabd1424 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:08.895719 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd831ad-97d8-48d2-b68c-1ebbaabd1424 None admin] PATH_INFO: `/v3/projects` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:08.896469 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd831ad-97d8-48d2-b68c-1ebbaabd1424 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:47:08.896774 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd831ad-97d8-48d2-b68c-1ebbaabd1424 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:08.898233 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd831ad-97d8-48d2-b68c-1ebbaabd1424 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:08.911042 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 419/850] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Aug 30 15:47:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:47:08.916822 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7920c7f4-ed4c-4e1d-89c7-8896faf26327 None None] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:08.917109 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7920c7f4-ed4c-4e1d-89c7-8896faf26327 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:08.917254 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd831ad-97d8-48d2-b68c-1ebbaabd1424 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:08.917419 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7920c7f4-ed4c-4e1d-89c7-8896faf26327 None None] PATH_INFO: `/` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:08.920119 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 432/851] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 30 15:47:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 15:47:08.924164 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e080f96-1822-4ced-9fcd-d25268ead8b3 None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:08.924441 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e080f96-1822-4ced-9fcd-d25268ead8b3 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:08.924706 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e080f96-1822-4ced-9fcd-d25268ead8b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:08.946425 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-2e080f96-1822-4ced-9fcd-d25268ead8b3 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:47:08.947815 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-2e080f96-1822-4ced-9fcd-d25268ead8b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:47:08.972869 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:47:09.037989 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 433/852] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 30 15:47:08 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 30 15:47:09.041282 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd831ad-97d8-48d2-b68c-1ebbaabd1424 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:09.054307 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:47:09.076648 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01e1c9b-9954-4592-93d6-c91e0ed0da89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:09.077593 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01e1c9b-9954-4592-93d6-c91e0ed0da89 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:09.078067 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01e1c9b-9954-4592-93d6-c91e0ed0da89 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:09.078482 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01e1c9b-9954-4592-93d6-c91e0ed0da89 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:09.079410 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01e1c9b-9954-4592-93d6-c91e0ed0da89 None admin] RBAC: Adding query filter params (role.id=f265386950a641ac8413b53eda4ae34d, scope.project.id=854fe71b49224dff9a4b1766a20ab266, user.id=8fdafdc39a594bb4b0aeaf2fbc84c2fb) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:47:09.079916 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01e1c9b-9954-4592-93d6-c91e0ed0da89 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:09.081651 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01e1c9b-9954-4592-93d6-c91e0ed0da89 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:09.089332 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 420/853] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Aug 30 15:47:08 2022] GET /identity/v3/role_assignments?user.id=8fdafdc39a594bb4b0aeaf2fbc84c2fb&scope.project.id=854fe71b49224dff9a4b1766a20ab266&role.id=f265386950a641ac8413b53eda4ae34d => generated 608 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 15:47:09.230769 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cfcc83-69bd-448d-8aad-e4b3bb2b8a9c None None] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:09.231014 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cfcc83-69bd-448d-8aad-e4b3bb2b8a9c None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:09.231457 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cfcc83-69bd-448d-8aad-e4b3bb2b8a9c None None] PATH_INFO: `/` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:09.233060 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 434/854] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 30 15:47:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 15:47:09.238450 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42b270a-bca7-4644-8dae-b381a5784c28 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:09.238948 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42b270a-bca7-4644-8dae-b381a5784c28 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:09.239407 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42b270a-bca7-4644-8dae-b381a5784c28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:09.259231 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-e42b270a-bca7-4644-8dae-b381a5784c28 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:47:09.260414 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-e42b270a-bca7-4644-8dae-b381a5784c28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:47:09.304275 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cfcc83-69bd-448d-8aad-e4b3bb2b8a9c None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:09.334837 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 421/855] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 30 15:47:09 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 15:47:09.419662 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cfcc83-69bd-448d-8aad-e4b3bb2b8a9c None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:09.455976 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec75ccdb-a594-44f8-8c9e-3eb873622ba8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:09.456610 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec75ccdb-a594-44f8-8c9e-3eb873622ba8 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:09.456887 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec75ccdb-a594-44f8-8c9e-3eb873622ba8 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:09.457155 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec75ccdb-a594-44f8-8c9e-3eb873622ba8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:09.467953 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec75ccdb-a594-44f8-8c9e-3eb873622ba8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 15:47:09.472762 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application [None req-ec75ccdb-a594-44f8-8c9e-3eb873622ba8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 15:47:09.472762 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:47:09.472762 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:47:09.472762 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:47:09.472762 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:47:09.472762 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:47:09.472762 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:47:09.472762 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:47:09.472762 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:47:09.472762 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:47:09.472762 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:47:09.472762 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:47:09.472762 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 15:47:09.472762 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 15:47:09.472762 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 15:47:09.472762 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 15:47:09.473742 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 15:47:09.473742 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 15:47:09.473742 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 15:47:09.473742 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 15:47:09.473742 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:47:09.473742 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:47:09.473742 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:47:09.473742 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:47:09.473742 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:47:09.473742 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:47:09.473742 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:47:09.473742 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application with Lock( Aug 30 15:47:09.473742 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:47:09.473742 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:47:09.473742 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:47:09.473742 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:47:09.474584 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:47:09.474584 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:47:09.474584 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:47:09.474584 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:47:09.474584 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 15:47:09.474584 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 15:47:09.474584 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 15:47:09.474584 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 15:47:09.474584 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 15:47:09.474584 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 15:47:09.474584 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 15:47:09.474584 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Aug 30 15:47:09.478857 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 435/856] 10.222.0.20 () {56 vars in 1102 bytes} [Tue Aug 30 15:47:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 15:47:09.480289 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42b270a-bca7-4644-8dae-b381a5784c28 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:09.582781 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42b270a-bca7-4644-8dae-b381a5784c28 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:09.591056 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:47:09.596533 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec75ccdb-a594-44f8-8c9e-3eb873622ba8 None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:09.619642 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:47:09.625060 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349196c7-defe-45e6-9af7-79e89e482736 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:09.626599 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349196c7-defe-45e6-9af7-79e89e482736 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:09.626881 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349196c7-defe-45e6-9af7-79e89e482736 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:09.627150 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349196c7-defe-45e6-9af7-79e89e482736 None admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:09.628010 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349196c7-defe-45e6-9af7-79e89e482736 None admin] RBAC: Adding query filter params (name=service) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:47:09.628311 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349196c7-defe-45e6-9af7-79e89e482736 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:09.629752 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349196c7-defe-45e6-9af7-79e89e482736 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:09.640377 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 422/857] 10.222.0.20 () {56 vars in 1103 bytes} [Tue Aug 30 15:47:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:47:09.652056 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349196c7-defe-45e6-9af7-79e89e482736 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:09.733548 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec75ccdb-a594-44f8-8c9e-3eb873622ba8 None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:09.769507 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044e59b5-dca7-4427-b58a-93256889219f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:09.770226 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044e59b5-dca7-4427-b58a-93256889219f None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:09.770566 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044e59b5-dca7-4427-b58a-93256889219f None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:09.771321 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044e59b5-dca7-4427-b58a-93256889219f None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:09.771724 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044e59b5-dca7-4427-b58a-93256889219f None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:09.773207 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044e59b5-dca7-4427-b58a-93256889219f None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:09.783109 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application [None req-044e59b5-dca7-4427-b58a-93256889219f None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 30 15:47:09.783109 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:47:09.783109 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 15:47:09.783109 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 15:47:09.783109 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 15:47:09.783109 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 15:47:09.783109 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 15:47:09.783109 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 15:47:09.783109 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 30 15:47:09.783109 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Aug 30 15:47:09.783109 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 15:47:09.783109 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Aug 30 15:47:09.783109 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:47:09.783109 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:47:09.783109 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:47:09.783109 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:47:09.784086 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:47:09.784086 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:47:09.784086 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:47:09.784086 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:47:09.784086 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:47:09.784086 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:47:09.784086 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:47:09.784086 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 15:47:09.784086 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 15:47:09.784086 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 15:47:09.784086 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 15:47:09.784086 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:47:09.784086 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:47:09.784086 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:47:09.784086 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:47:09.784086 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:47:09.784086 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:47:09.785010 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:47:09.785010 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application with Lock( Aug 30 15:47:09.785010 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:47:09.785010 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:47:09.785010 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:47:09.785010 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:47:09.785010 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:47:09.785010 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:47:09.785010 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:47:09.785010 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:47:09.785010 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 15:47:09.785010 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 15:47:09.785010 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 30 15:47:09.785010 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Aug 30 15:47:09.787733 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349196c7-defe-45e6-9af7-79e89e482736 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:09.789407 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 436/858] 10.222.0.20 () {56 vars in 1132 bytes} [Tue Aug 30 15:47:09 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 15:47:09.790420 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044e59b5-dca7-4427-b58a-93256889219f None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:09.824437 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950013c6-55ae-4630-925c-38f953e37a4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:09.825203 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950013c6-55ae-4630-925c-38f953e37a4b None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:09.825584 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950013c6-55ae-4630-925c-38f953e37a4b None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:09.825916 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950013c6-55ae-4630-925c-38f953e37a4b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:09.837752 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950013c6-55ae-4630-925c-38f953e37a4b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:09.839400 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950013c6-55ae-4630-925c-38f953e37a4b None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:09.854643 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-950013c6-55ae-4630-925c-38f953e37a4b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 15:47:09.854643 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:47:09.854643 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:47:09.854643 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:47:09.854643 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:47:09.854643 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:47:09.854643 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:47:09.854643 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:47:09.854643 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:47:09.854643 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:47:09.854643 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:47:09.854643 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:47:09.854643 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 15:47:09.854643 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 15:47:09.854643 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 15:47:09.854643 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 15:47:09.855655 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:47:09.855655 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:47:09.855655 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:47:09.855655 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:47:09.855655 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:47:09.855655 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:47:09.855655 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:47:09.855655 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Aug 30 15:47:09.855655 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:47:09.855655 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:47:09.855655 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:47:09.855655 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:47:09.855655 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:47:09.855655 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:47:09.855655 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:47:09.855655 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:47:09.855655 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 15:47:09.856569 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 15:47:09.856569 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 15:47:09.856569 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Aug 30 15:47:09.860861 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 423/859] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 30 15:47:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 15:47:09.861749 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950013c6-55ae-4630-925c-38f953e37a4b None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:09.885461 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:47:09.903729 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:47:09.928776 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044e59b5-dca7-4427-b58a-93256889219f None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:09.968744 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950013c6-55ae-4630-925c-38f953e37a4b None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:09.969790 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cf6d7b-dea5-4c8e-96a6-942cd128c3bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:09.970403 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cf6d7b-dea5-4c8e-96a6-942cd128c3bd None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:09.970687 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cf6d7b-dea5-4c8e-96a6-942cd128c3bd None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:09.971653 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cf6d7b-dea5-4c8e-96a6-942cd128c3bd None admin] PATH_INFO: `/v3/services` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:09.971653 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05cf6d7b-dea5-4c8e-96a6-942cd128c3bd None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81512) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:47:09.971902 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05cf6d7b-dea5-4c8e-96a6-942cd128c3bd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:09.973284 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05cf6d7b-dea5-4c8e-96a6-942cd128c3bd None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:09.981803 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 437/860] 10.222.0.20 () {56 vars in 1126 bytes} [Tue Aug 30 15:47:09 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:47:09.981803 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3daeac-78c4-42ef-9456-b1690b716167 None None] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:09.981803 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3daeac-78c4-42ef-9456-b1690b716167 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:09.981803 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3daeac-78c4-42ef-9456-b1690b716167 None None] PATH_INFO: `/` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:09.981803 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 438/861] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 30 15:47:09 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 15:47:09.985395 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3daeac-78c4-42ef-9456-b1690b716167 None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:10.009844 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc46fce4-8057-4136-ac91-5fbcb21f631b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:10.010861 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc46fce4-8057-4136-ac91-5fbcb21f631b None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:10.011380 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc46fce4-8057-4136-ac91-5fbcb21f631b None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:10.011834 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc46fce4-8057-4136-ac91-5fbcb21f631b None admin] PATH_INFO: `/v3/domains` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:10.012615 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc46fce4-8057-4136-ac91-5fbcb21f631b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:47:10.013152 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc46fce4-8057-4136-ac91-5fbcb21f631b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:10.014807 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc46fce4-8057-4136-ac91-5fbcb21f631b None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:10.027731 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 424/862] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Aug 30 15:47:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:47:10.029354 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4240699-6968-4325-a696-ec00b1a691f1 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:10.029691 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4240699-6968-4325-a696-ec00b1a691f1 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:10.029959 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4240699-6968-4325-a696-ec00b1a691f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:10.048196 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-f4240699-6968-4325-a696-ec00b1a691f1 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:47:10.049409 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-f4240699-6968-4325-a696-ec00b1a691f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:47:10.097576 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3daeac-78c4-42ef-9456-b1690b716167 None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:10.123962 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 425/863] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 30 15:47:10 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 15:47:10.124833 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4240699-6968-4325-a696-ec00b1a691f1 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:10.137087 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:47:10.147002 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd7a7b5-0e4a-43ef-8cf7-b353e017509c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:10.148059 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd7a7b5-0e4a-43ef-8cf7-b353e017509c None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:10.148562 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd7a7b5-0e4a-43ef-8cf7-b353e017509c None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:10.149498 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd7a7b5-0e4a-43ef-8cf7-b353e017509c None admin] PATH_INFO: `/v3/services` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:10.150612 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd7a7b5-0e4a-43ef-8cf7-b353e017509c None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81512) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:47:10.151112 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd7a7b5-0e4a-43ef-8cf7-b353e017509c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:10.153504 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd7a7b5-0e4a-43ef-8cf7-b353e017509c None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:10.160293 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 439/864] 10.222.0.20 () {56 vars in 1126 bytes} [Tue Aug 30 15:47:09 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:47:10.243712 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4240699-6968-4325-a696-ec00b1a691f1 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:10.286570 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099ba304-4847-4759-91c0-e674249cac30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:10.287246 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099ba304-4847-4759-91c0-e674249cac30 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:10.287519 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099ba304-4847-4759-91c0-e674249cac30 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:10.287783 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099ba304-4847-4759-91c0-e674249cac30 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:10.296571 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099ba304-4847-4759-91c0-e674249cac30 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:10.298139 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099ba304-4847-4759-91c0-e674249cac30 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:10.312548 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-099ba304-4847-4759-91c0-e674249cac30 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 30 15:47:10.312548 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:47:10.312548 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:47:10.312548 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:47:10.312548 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:47:10.312548 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:47:10.312548 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:47:10.312548 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:47:10.312548 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:47:10.312548 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:47:10.312548 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:47:10.312548 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:47:10.312548 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 15:47:10.312548 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 15:47:10.312548 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 15:47:10.312548 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 15:47:10.313605 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:47:10.313605 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:47:10.313605 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 15:47:10.313605 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 15:47:10.313605 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 15:47:10.313605 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 15:47:10.313605 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:47:10.313605 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:47:10.313605 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:47:10.313605 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:47:10.313605 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:47:10.313605 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Aug 30 15:47:10.313605 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:47:10.313605 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:47:10.313605 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:47:10.313605 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:47:10.313605 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:47:10.313605 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:47:10.314556 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:47:10.314556 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:47:10.314556 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 15:47:10.314556 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 15:47:10.314556 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 15:47:10.314556 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 15:47:10.314556 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 15:47:10.314556 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 15:47:10.314556 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 30 15:47:10.314556 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Aug 30 15:47:10.314556 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 426/865] 10.222.0.20 () {56 vars in 1117 bytes} [Tue Aug 30 15:47:10 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 15:47:10.319764 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd7a7b5-0e4a-43ef-8cf7-b353e017509c None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:10.339539 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:47:10.405563 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099ba304-4847-4759-91c0-e674249cac30 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:10.440273 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd7a7b5-0e4a-43ef-8cf7-b353e017509c None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:10.476352 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7187ce12-96d0-4e24-8c3e-e871112cc816 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:10.476977 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7187ce12-96d0-4e24-8c3e-e871112cc816 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:10.477274 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7187ce12-96d0-4e24-8c3e-e871112cc816 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:10.477548 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7187ce12-96d0-4e24-8c3e-e871112cc816 None admin] PATH_INFO: `/v3/users` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:10.478281 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7187ce12-96d0-4e24-8c3e-e871112cc816 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81512) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:47:10.478585 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7187ce12-96d0-4e24-8c3e-e871112cc816 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:10.480036 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7187ce12-96d0-4e24-8c3e-e871112cc816 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:10.483139 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:47:10.500635 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 440/866] 10.222.0.20 () {56 vars in 1149 bytes} [Tue Aug 30 15:47:10 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:47:10.507212 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7187ce12-96d0-4e24-8c3e-e871112cc816 None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:10.524846 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099ba304-4847-4759-91c0-e674249cac30 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:10.561894 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d14ab3-0280-46c4-9476-d64c128719cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:10.562884 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d14ab3-0280-46c4-9476-d64c128719cc None admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:10.563349 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d14ab3-0280-46c4-9476-d64c128719cc None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:10.563800 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d14ab3-0280-46c4-9476-d64c128719cc None admin] PATH_INFO: `/v3/services` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:10.564643 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d14ab3-0280-46c4-9476-d64c128719cc None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:10.566262 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d14ab3-0280-46c4-9476-d64c128719cc None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:10.585812 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 427/867] 10.222.0.20 () {60 vars in 1141 bytes} [Tue Aug 30 15:47:10 2022] POST /identity/v3/services => generated 251 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 15:47:10.586433 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:47:10.653249 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7187ce12-96d0-4e24-8c3e-e871112cc816 None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:10.706011 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a5bacf-934b-4484-b2fa-eae800d7fd6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:10.706559 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a5bacf-934b-4484-b2fa-eae800d7fd6b None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:10.706833 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a5bacf-934b-4484-b2fa-eae800d7fd6b None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:10.707098 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a5bacf-934b-4484-b2fa-eae800d7fd6b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:10.782897 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a5bacf-934b-4484-b2fa-eae800d7fd6b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:10.784362 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a5bacf-934b-4484-b2fa-eae800d7fd6b None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:10.852729 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:47:10.895883 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 441/868] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Aug 30 15:47:10 2022] GET /identity/v3/auth/tokens => generated 1262 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 30 15:47:10.901409 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d14ab3-0280-46c4-9476-d64c128719cc None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:11.007826 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d14ab3-0280-46c4-9476-d64c128719cc None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:11.037066 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:47:11.042839 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50b4521-cf92-486b-91e6-dd9784587759 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:11.043519 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50b4521-cf92-486b-91e6-dd9784587759 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:11.043793 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50b4521-cf92-486b-91e6-dd9784587759 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:11.044069 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50b4521-cf92-486b-91e6-dd9784587759 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:11.056292 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50b4521-cf92-486b-91e6-dd9784587759 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:11.057800 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50b4521-cf92-486b-91e6-dd9784587759 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:11.072835 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-b50b4521-cf92-486b-91e6-dd9784587759 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 15:47:11.072835 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:47:11.072835 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:47:11.072835 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:47:11.072835 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:47:11.072835 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:47:11.072835 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:47:11.072835 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:47:11.072835 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:47:11.072835 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:47:11.072835 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:47:11.072835 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:47:11.072835 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 15:47:11.072835 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 15:47:11.072835 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 15:47:11.072835 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 15:47:11.075117 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:47:11.075117 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:47:11.075117 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:47:11.075117 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:47:11.075117 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:47:11.075117 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:47:11.075117 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:47:11.075117 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Aug 30 15:47:11.075117 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:47:11.075117 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:47:11.075117 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:47:11.075117 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:47:11.075117 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:47:11.075117 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:47:11.075117 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:47:11.075117 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:47:11.075117 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 15:47:11.077383 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 15:47:11.077383 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 15:47:11.077383 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Aug 30 15:47:11.077383 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 428/869] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 30 15:47:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 15:47:11.082623 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a5bacf-934b-4484-b2fa-eae800d7fd6b None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:11.191115 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883cb413-ebd6-4d87-b1a6-3b290ec5be28 None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:11.191396 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883cb413-ebd6-4d87-b1a6-3b290ec5be28 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:11.191669 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883cb413-ebd6-4d87-b1a6-3b290ec5be28 None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:11.197119 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 429/870] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 30 15:47:11 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 15:47:11.198660 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6e8ea6-0829-4726-9717-088f8f98d492 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:11.198945 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6e8ea6-0829-4726-9717-088f8f98d492 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:11.199217 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6e8ea6-0829-4726-9717-088f8f98d492 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:11.217441 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-3c6e8ea6-0829-4726-9717-088f8f98d492 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:47:11.218827 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-3c6e8ea6-0829-4726-9717-088f8f98d492 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:47:11.219700 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a5bacf-934b-4484-b2fa-eae800d7fd6b None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:11.263355 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fc7420-13ea-4351-a5ff-5e2b1b3555b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:11.263911 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fc7420-13ea-4351-a5ff-5e2b1b3555b3 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:11.264187 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fc7420-13ea-4351-a5ff-5e2b1b3555b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:11.264448 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fc7420-13ea-4351-a5ff-5e2b1b3555b3 None admin] PATH_INFO: `/v3/domains` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:11.265476 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fc7420-13ea-4351-a5ff-5e2b1b3555b3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81512) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:47:11.265772 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fc7420-13ea-4351-a5ff-5e2b1b3555b3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:11.267745 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fc7420-13ea-4351-a5ff-5e2b1b3555b3 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:11.283689 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 442/871] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Aug 30 15:47:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 15:47:11.289436 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fc7420-13ea-4351-a5ff-5e2b1b3555b3 None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:11.297335 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 430/872] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 30 15:47:11 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 30 15:47:11.314724 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:47:11.391260 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fc7420-13ea-4351-a5ff-5e2b1b3555b3 None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:11.426279 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c99787-5a34-490f-ad2f-506d8de4437f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:11.426907 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c99787-5a34-490f-ad2f-506d8de4437f None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:11.427179 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c99787-5a34-490f-ad2f-506d8de4437f None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:11.427441 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c99787-5a34-490f-ad2f-506d8de4437f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:11.434055 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c99787-5a34-490f-ad2f-506d8de4437f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:11.435498 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c99787-5a34-490f-ad2f-506d8de4437f None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:11.446820 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application [None req-67c99787-5a34-490f-ad2f-506d8de4437f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 15:47:11.446820 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:47:11.446820 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:47:11.446820 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:47:11.446820 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:47:11.446820 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:47:11.446820 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:47:11.446820 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:47:11.446820 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:47:11.446820 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:47:11.446820 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:47:11.446820 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:47:11.446820 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 15:47:11.446820 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 15:47:11.446820 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 15:47:11.448141 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 15:47:11.448141 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:47:11.448141 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:47:11.448141 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:47:11.448141 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:47:11.448141 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:47:11.448141 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:47:11.448141 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:47:11.448141 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application with Lock( Aug 30 15:47:11.448141 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:47:11.448141 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:47:11.448141 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:47:11.448141 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:47:11.448141 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:47:11.448141 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:47:11.448141 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:47:11.448141 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:47:11.449117 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 15:47:11.449117 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 15:47:11.449117 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 15:47:11.449117 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 15:47:11.449117 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 15:47:11.449117 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 15:47:11.449117 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 15:47:11.449117 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Aug 30 15:47:11.449117 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 443/873] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Aug 30 15:47:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 15:47:11.458326 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6e8ea6-0829-4726-9717-088f8f98d492 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:11.558003 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6e8ea6-0829-4726-9717-088f8f98d492 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:11.575423 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:47:11.577501 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c99787-5a34-490f-ad2f-506d8de4437f None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:11.597087 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdbcf9d-d2f6-46e3-8e10-5df369c7de3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:11.597743 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdbcf9d-d2f6-46e3-8e10-5df369c7de3c None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:11.598023 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdbcf9d-d2f6-46e3-8e10-5df369c7de3c None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:11.598413 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdbcf9d-d2f6-46e3-8e10-5df369c7de3c None admin] PATH_INFO: `/v3/projects` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:11.599115 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fdbcf9d-d2f6-46e3-8e10-5df369c7de3c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:47:11.599414 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fdbcf9d-d2f6-46e3-8e10-5df369c7de3c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:11.600852 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fdbcf9d-d2f6-46e3-8e10-5df369c7de3c None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:11.617883 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 431/874] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Aug 30 15:47:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:47:11.619053 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdbcf9d-d2f6-46e3-8e10-5df369c7de3c None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:11.683040 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:47:11.707278 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c99787-5a34-490f-ad2f-506d8de4437f None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:11.744464 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdbcf9d-d2f6-46e3-8e10-5df369c7de3c None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:11.760594 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65ad786-8c32-4d33-9ee8-d2a36631095e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:11.761261 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65ad786-8c32-4d33-9ee8-d2a36631095e None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:11.761547 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65ad786-8c32-4d33-9ee8-d2a36631095e None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:11.761819 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65ad786-8c32-4d33-9ee8-d2a36631095e None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:11.762570 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65ad786-8c32-4d33-9ee8-d2a36631095e None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:11.764583 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65ad786-8c32-4d33-9ee8-d2a36631095e None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:11.775783 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application [None req-c65ad786-8c32-4d33-9ee8-d2a36631095e None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 30 15:47:11.775783 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:47:11.775783 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 15:47:11.775783 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 15:47:11.775783 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 15:47:11.775783 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 15:47:11.775783 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 15:47:11.775783 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 15:47:11.775783 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 30 15:47:11.775783 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Aug 30 15:47:11.775783 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 15:47:11.775783 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Aug 30 15:47:11.775783 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:47:11.775783 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:47:11.775783 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:47:11.775783 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:47:11.775783 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:47:11.776802 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:47:11.776802 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:47:11.776802 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:47:11.776802 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:47:11.776802 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:47:11.776802 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:47:11.776802 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 15:47:11.776802 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 15:47:11.776802 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 15:47:11.776802 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 15:47:11.776802 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:47:11.776802 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:47:11.776802 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:47:11.776802 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:47:11.776802 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:47:11.776802 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:47:11.777676 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:47:11.777676 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application with Lock( Aug 30 15:47:11.777676 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:47:11.777676 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:47:11.777676 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:47:11.777676 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:47:11.777676 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:47:11.777676 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:47:11.777676 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:47:11.777676 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:47:11.777676 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 15:47:11.777676 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 15:47:11.777676 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 30 15:47:11.777676 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Aug 30 15:47:11.781297 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 444/875] 10.222.0.20 () {56 vars in 1126 bytes} [Tue Aug 30 15:47:11 2022] GET /identity/v3/services/object-store => generated 93 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 15:47:11.782857 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65ad786-8c32-4d33-9ee8-d2a36631095e None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:11.796646 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715af658-f86e-41f2-aa2e-be2c7b04d7bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:11.797300 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715af658-f86e-41f2-aa2e-be2c7b04d7bc None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:11.797574 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715af658-f86e-41f2-aa2e-be2c7b04d7bc None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:11.797842 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715af658-f86e-41f2-aa2e-be2c7b04d7bc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:11.798935 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715af658-f86e-41f2-aa2e-be2c7b04d7bc None admin] RBAC: Adding query filter params (role.id=57eb8466096346b39319536c6feeb6f7, scope.project.id=854fe71b49224dff9a4b1766a20ab266, user.id=07130d598131443486c2bc1c6673c45b) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:47:11.799241 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715af658-f86e-41f2-aa2e-be2c7b04d7bc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:11.801112 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715af658-f86e-41f2-aa2e-be2c7b04d7bc None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:11.817466 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 432/876] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Aug 30 15:47:11 2022] GET /identity/v3/role_assignments?user.id=07130d598131443486c2bc1c6673c45b&scope.project.id=854fe71b49224dff9a4b1766a20ab266&role.id=57eb8466096346b39319536c6feeb6f7 => generated 608 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 15:47:11.907795 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65ad786-8c32-4d33-9ee8-d2a36631095e None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:11.947060 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7930dbfc-6713-4a5f-9d08-0696380105da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:11.948082 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7930dbfc-6713-4a5f-9d08-0696380105da None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:11.948624 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7930dbfc-6713-4a5f-9d08-0696380105da None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:11.949100 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7930dbfc-6713-4a5f-9d08-0696380105da None admin] PATH_INFO: `/v3/services` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:11.949989 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7930dbfc-6713-4a5f-9d08-0696380105da None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81512) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:47:11.950485 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7930dbfc-6713-4a5f-9d08-0696380105da None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:11.952104 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7930dbfc-6713-4a5f-9d08-0696380105da None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:11.957928 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 445/877] 10.222.0.20 () {56 vars in 1122 bytes} [Tue Aug 30 15:47:11 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:47:11.964449 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715af658-f86e-41f2-aa2e-be2c7b04d7bc None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:12.006588 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:47:12.076244 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715af658-f86e-41f2-aa2e-be2c7b04d7bc None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:12.110593 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93617cf3-d97f-41c8-8f89-146274cff39c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:12.111199 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93617cf3-d97f-41c8-8f89-146274cff39c None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:12.111469 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93617cf3-d97f-41c8-8f89-146274cff39c None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:12.111733 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93617cf3-d97f-41c8-8f89-146274cff39c None admin] PATH_INFO: `/v3/services` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:12.112361 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93617cf3-d97f-41c8-8f89-146274cff39c None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:47:12.112653 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93617cf3-d97f-41c8-8f89-146274cff39c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:12.114196 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93617cf3-d97f-41c8-8f89-146274cff39c None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:12.119164 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 433/878] 10.222.0.20 () {56 vars in 1122 bytes} [Tue Aug 30 15:47:11 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:47:12.242567 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520b337e-e22e-41f9-bde8-bf1ee9fc679b None None] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:12.242966 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520b337e-e22e-41f9-bde8-bf1ee9fc679b None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:12.243147 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520b337e-e22e-41f9-bde8-bf1ee9fc679b None None] PATH_INFO: `/` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:12.244735 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 446/879] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 30 15:47:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 15:47:12.250931 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549cd4b7-256c-419c-a0f1-b05b96d77b9c None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:12.251546 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549cd4b7-256c-419c-a0f1-b05b96d77b9c None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:12.252025 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549cd4b7-256c-419c-a0f1-b05b96d77b9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:12.273035 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-549cd4b7-256c-419c-a0f1-b05b96d77b9c None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:47:12.274040 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-549cd4b7-256c-419c-a0f1-b05b96d77b9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:47:12.343124 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 434/880] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 30 15:47:12 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 30 15:47:12.615273 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520b337e-e22e-41f9-bde8-bf1ee9fc679b None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:12.629004 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:47:12.665075 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98cea69-976c-404f-92e8-77b3c07c9d69 None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:12.665387 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98cea69-976c-404f-92e8-77b3c07c9d69 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:12.665642 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98cea69-976c-404f-92e8-77b3c07c9d69 None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:12.667202 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 435/881] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 30 15:47:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 15:47:12.672440 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09cb00b-9195-4584-9d64-ebfc1bd8b799 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:12.672711 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09cb00b-9195-4584-9d64-ebfc1bd8b799 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:12.672974 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09cb00b-9195-4584-9d64-ebfc1bd8b799 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:12.694342 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-f09cb00b-9195-4584-9d64-ebfc1bd8b799 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:47:12.695998 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-f09cb00b-9195-4584-9d64-ebfc1bd8b799 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:47:12.721655 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520b337e-e22e-41f9-bde8-bf1ee9fc679b None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:12.726406 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:47:12.768260 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c542e283-be23-4fec-9657-fd58a441f72d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:12.769285 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c542e283-be23-4fec-9657-fd58a441f72d None admin] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:12.769565 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c542e283-be23-4fec-9657-fd58a441f72d None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:12.770082 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c542e283-be23-4fec-9657-fd58a441f72d None admin] PATH_INFO: `/v3/services` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:12.770744 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c542e283-be23-4fec-9657-fd58a441f72d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:12.773040 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c542e283-be23-4fec-9657-fd58a441f72d None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:12.779720 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 436/882] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 30 15:47:12 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 30 15:47:12.794958 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 447/883] 10.222.0.20 () {60 vars in 1141 bytes} [Tue Aug 30 15:47:12 2022] POST /identity/v3/services => generated 269 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 15:47:13.047047 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09cb00b-9195-4584-9d64-ebfc1bd8b799 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:13.152417 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09cb00b-9195-4584-9d64-ebfc1bd8b799 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:13.192477 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2205750-6adf-4e10-b7ee-5fc917eb0331 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:13.193105 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2205750-6adf-4e10-b7ee-5fc917eb0331 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:13.193405 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2205750-6adf-4e10-b7ee-5fc917eb0331 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:13.193668 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2205750-6adf-4e10-b7ee-5fc917eb0331 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:13.194248 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2205750-6adf-4e10-b7ee-5fc917eb0331 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:13.195669 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2205750-6adf-4e10-b7ee-5fc917eb0331 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:13.204465 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-b2205750-6adf-4e10-b7ee-5fc917eb0331 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 30 15:47:13.204465 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:47:13.204465 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 15:47:13.204465 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 15:47:13.204465 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 15:47:13.204465 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 15:47:13.204465 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 15:47:13.204465 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 15:47:13.204465 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 30 15:47:13.204465 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Aug 30 15:47:13.204465 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 15:47:13.204465 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Aug 30 15:47:13.204465 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:47:13.204465 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:47:13.204465 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:47:13.204465 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:47:13.204465 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:47:13.206800 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:47:13.206800 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:47:13.206800 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:47:13.206800 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:47:13.206800 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:47:13.206800 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:47:13.206800 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 15:47:13.206800 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 15:47:13.206800 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 15:47:13.206800 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 15:47:13.206800 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:47:13.206800 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:47:13.206800 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:47:13.206800 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:47:13.206800 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:47:13.206800 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:47:13.208812 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:47:13.208812 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Aug 30 15:47:13.208812 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:47:13.208812 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:47:13.208812 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:47:13.208812 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:47:13.208812 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:47:13.208812 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:47:13.208812 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:47:13.208812 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:47:13.208812 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 15:47:13.208812 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 15:47:13.208812 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 30 15:47:13.208812 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Aug 30 15:47:13.208812 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 437/884] 10.222.0.20 () {56 vars in 1114 bytes} [Tue Aug 30 15:47:13 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 15:47:13.213509 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c542e283-be23-4fec-9657-fd58a441f72d None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:13.292410 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:47:13.318779 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c542e283-be23-4fec-9657-fd58a441f72d None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:13.353285 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3d9d64-ea3d-4a75-b86d-49e9b22cbf53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:13.353903 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3d9d64-ea3d-4a75-b86d-49e9b22cbf53 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:13.354182 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3d9d64-ea3d-4a75-b86d-49e9b22cbf53 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:13.354452 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3d9d64-ea3d-4a75-b86d-49e9b22cbf53 None admin] PATH_INFO: `/v3/services` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:13.355085 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3d9d64-ea3d-4a75-b86d-49e9b22cbf53 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81512) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:47:13.355388 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3d9d64-ea3d-4a75-b86d-49e9b22cbf53 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:13.356771 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3d9d64-ea3d-4a75-b86d-49e9b22cbf53 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:13.361886 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 448/885] 10.222.0.20 () {56 vars in 1114 bytes} [Tue Aug 30 15:47:13 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:47:13.371079 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2205750-6adf-4e10-b7ee-5fc917eb0331 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:13.394833 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:47:13.475586 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2205750-6adf-4e10-b7ee-5fc917eb0331 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:13.511288 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5cb866-f980-46ca-b27d-e546d54c2c67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:13.512306 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5cb866-f980-46ca-b27d-e546d54c2c67 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:13.512811 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5cb866-f980-46ca-b27d-e546d54c2c67 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:13.513290 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5cb866-f980-46ca-b27d-e546d54c2c67 None admin] PATH_INFO: `/v3/services` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:13.514158 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5cb866-f980-46ca-b27d-e546d54c2c67 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:47:13.514656 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5cb866-f980-46ca-b27d-e546d54c2c67 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:13.516197 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5cb866-f980-46ca-b27d-e546d54c2c67 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:13.521729 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 438/886] 10.222.0.20 () {56 vars in 1114 bytes} [Tue Aug 30 15:47:13 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:47:13.527701 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3d9d64-ea3d-4a75-b86d-49e9b22cbf53 None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:13.630349 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3d9d64-ea3d-4a75-b86d-49e9b22cbf53 None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:13.665751 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289d2f7b-83d3-4ae1-bfcd-a5c53b7a85b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:13.666354 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289d2f7b-83d3-4ae1-bfcd-a5c53b7a85b3 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:13.666626 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289d2f7b-83d3-4ae1-bfcd-a5c53b7a85b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:13.666891 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289d2f7b-83d3-4ae1-bfcd-a5c53b7a85b3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:13.667590 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289d2f7b-83d3-4ae1-bfcd-a5c53b7a85b3 None admin] RBAC: Adding query filter params (interface=public, service_id=1fc3c58024c748c6978fe04214545736, region_id=RegionOne) {{(pid=81512) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:47:13.667892 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289d2f7b-83d3-4ae1-bfcd-a5c53b7a85b3 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:13.669278 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289d2f7b-83d3-4ae1-bfcd-a5c53b7a85b3 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:13.670875 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:47:13.677129 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 449/887] 10.222.0.20 () {56 vars in 1251 bytes} [Tue Aug 30 15:47:13 2022] GET /identity/v3/endpoints?service_id=1fc3c58024c748c6978fe04214545736&interface=public®ion_id=RegionOne => generated 194 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:47:13.682781 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5cb866-f980-46ca-b27d-e546d54c2c67 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:13.769893 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:47:13.786613 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5cb866-f980-46ca-b27d-e546d54c2c67 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:13.820516 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf76a827-3163-4e83-be39-5739e39e5aff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:13.821128 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf76a827-3163-4e83-be39-5739e39e5aff None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:13.821471 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf76a827-3163-4e83-be39-5739e39e5aff None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:13.821739 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf76a827-3163-4e83-be39-5739e39e5aff None admin] PATH_INFO: `/v3/services` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:13.822339 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf76a827-3163-4e83-be39-5739e39e5aff None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:13.823706 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf76a827-3163-4e83-be39-5739e39e5aff None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:13.830075 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 439/888] 10.222.0.20 () {56 vars in 1087 bytes} [Tue Aug 30 15:47:13 2022] GET /identity/v3/services => generated 1268 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 15:47:13.884888 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4c7fc1-1a32-4f56-a23d-8df22fcfa917 None None] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:13.885209 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4c7fc1-1a32-4f56-a23d-8df22fcfa917 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:13.885395 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4c7fc1-1a32-4f56-a23d-8df22fcfa917 None None] PATH_INFO: `/` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:13.887082 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 450/889] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 30 15:47:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 15:47:13.892630 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6fe5e3-67c4-4d4f-b67f-7889f191324f None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:13.892893 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6fe5e3-67c4-4d4f-b67f-7889f191324f None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:13.893165 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6fe5e3-67c4-4d4f-b67f-7889f191324f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:13.913315 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-4c6fe5e3-67c4-4d4f-b67f-7889f191324f None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:47:13.914555 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-4c6fe5e3-67c4-4d4f-b67f-7889f191324f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:47:13.994052 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 440/890] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 30 15:47:13 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 15:47:14.083293 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b39e51-7d99-4d01-9641-d7bfe5c4a2a1 None None] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:14.083972 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b39e51-7d99-4d01-9641-d7bfe5c4a2a1 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:14.084456 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b39e51-7d99-4d01-9641-d7bfe5c4a2a1 None None] PATH_INFO: `/` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:14.086336 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 451/891] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 30 15:47:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 15:47:14.091884 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b56d077-e205-4f91-bb6f-a0f2c1fa2ed7 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:14.092805 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b56d077-e205-4f91-bb6f-a0f2c1fa2ed7 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:14.093343 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b56d077-e205-4f91-bb6f-a0f2c1fa2ed7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:14.112711 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-5b56d077-e205-4f91-bb6f-a0f2c1fa2ed7 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:47:14.114114 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-5b56d077-e205-4f91-bb6f-a0f2c1fa2ed7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:47:14.181482 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 441/892] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 30 15:47:14 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 15:47:14.258327 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b39e51-7d99-4d01-9641-d7bfe5c4a2a1 None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:14.360867 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b39e51-7d99-4d01-9641-d7bfe5c4a2a1 None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:14.395281 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d758314-4fc6-4770-a016-58575fcd16bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:14.395946 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d758314-4fc6-4770-a016-58575fcd16bd None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:14.396229 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d758314-4fc6-4770-a016-58575fcd16bd None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:14.396499 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d758314-4fc6-4770-a016-58575fcd16bd None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:14.411989 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d758314-4fc6-4770-a016-58575fcd16bd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 15:47:14.416844 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application [None req-1d758314-4fc6-4770-a016-58575fcd16bd None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 30 15:47:14.416844 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:47:14.416844 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:47:14.416844 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:47:14.416844 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:47:14.416844 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:47:14.416844 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:47:14.416844 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:47:14.416844 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:47:14.416844 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:47:14.416844 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:47:14.416844 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:47:14.416844 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 15:47:14.416844 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 15:47:14.416844 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 15:47:14.416844 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 15:47:14.417782 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 15:47:14.417782 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 15:47:14.417782 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 15:47:14.417782 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 15:47:14.417782 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:47:14.417782 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:47:14.417782 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:47:14.417782 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:47:14.417782 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:47:14.417782 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:47:14.417782 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:47:14.417782 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application with Lock( Aug 30 15:47:14.417782 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:47:14.417782 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:47:14.417782 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:47:14.417782 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:47:14.419154 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:47:14.419154 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:47:14.419154 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:47:14.419154 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:47:14.419154 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 15:47:14.419154 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 15:47:14.419154 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 15:47:14.419154 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 15:47:14.419154 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 15:47:14.419154 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 15:47:14.419154 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 30 15:47:14.419154 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Aug 30 15:47:14.419154 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 452/893] 10.222.0.20 () {56 vars in 1120 bytes} [Tue Aug 30 15:47:14 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 15:47:14.424126 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b56d077-e205-4f91-bb6f-a0f2c1fa2ed7 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:14.434873 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:47:14.449487 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d758314-4fc6-4770-a016-58575fcd16bd None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:14.532933 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b56d077-e205-4f91-bb6f-a0f2c1fa2ed7 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:14.570064 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c07177-6b67-4da4-b341-9d3559776896 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:14.570740 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c07177-6b67-4da4-b341-9d3559776896 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:14.571014 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c07177-6b67-4da4-b341-9d3559776896 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:14.571278 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c07177-6b67-4da4-b341-9d3559776896 None admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:14.571998 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:47:14.573048 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c07177-6b67-4da4-b341-9d3559776896 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:47:14.573407 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c07177-6b67-4da4-b341-9d3559776896 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:14.575005 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c07177-6b67-4da4-b341-9d3559776896 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:14.578713 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d758314-4fc6-4770-a016-58575fcd16bd None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:14.588281 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 442/894] 10.222.0.20 () {56 vars in 1115 bytes} [Tue Aug 30 15:47:14 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:47:14.594146 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c07177-6b67-4da4-b341-9d3559776896 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:14.622213 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e6986b-298c-4d48-a55c-a192c113297d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:14.623430 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e6986b-298c-4d48-a55c-a192c113297d None admin] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:14.624164 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e6986b-298c-4d48-a55c-a192c113297d None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:14.625050 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e6986b-298c-4d48-a55c-a192c113297d None admin] PATH_INFO: `/v3/services` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:14.632552 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e6986b-298c-4d48-a55c-a192c113297d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:14.635767 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e6986b-298c-4d48-a55c-a192c113297d None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:14.669527 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 453/895] 10.222.0.20 () {60 vars in 1141 bytes} [Tue Aug 30 15:47:14 2022] POST /identity/v3/services => generated 241 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 15:47:14.693701 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:47:14.721889 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c07177-6b67-4da4-b341-9d3559776896 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:14.773866 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0263b4-7c5d-48c8-8bad-9146921a0faf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:14.774927 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0263b4-7c5d-48c8-8bad-9146921a0faf None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:14.775433 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0263b4-7c5d-48c8-8bad-9146921a0faf None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:14.775901 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0263b4-7c5d-48c8-8bad-9146921a0faf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:14.788323 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0263b4-7c5d-48c8-8bad-9146921a0faf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:14.790191 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0263b4-7c5d-48c8-8bad-9146921a0faf None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:14.806775 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-fb0263b4-7c5d-48c8-8bad-9146921a0faf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 15:47:14.806775 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:47:14.806775 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:47:14.806775 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:47:14.806775 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:47:14.806775 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:47:14.806775 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:47:14.806775 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:47:14.806775 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:47:14.806775 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:47:14.806775 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:47:14.806775 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:47:14.806775 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 15:47:14.806775 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 15:47:14.806775 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 15:47:14.806775 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 15:47:14.807807 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:47:14.807807 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:47:14.807807 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:47:14.807807 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:47:14.807807 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:47:14.807807 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:47:14.807807 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:47:14.807807 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Aug 30 15:47:14.807807 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:47:14.807807 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:47:14.807807 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:47:14.807807 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:47:14.807807 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:47:14.807807 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:47:14.807807 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:47:14.807807 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:47:14.807807 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 15:47:14.808716 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 15:47:14.808716 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 15:47:14.808716 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Aug 30 15:47:14.811120 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 443/896] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 30 15:47:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 15:47:14.817437 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e6986b-298c-4d48-a55c-a192c113297d None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:14.896114 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1011884b-be45-4ff1-a40b-e6a23a3fbf6c None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:14.896490 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1011884b-be45-4ff1-a40b-e6a23a3fbf6c None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:14.896656 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1011884b-be45-4ff1-a40b-e6a23a3fbf6c None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:14.898280 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 444/897] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 30 15:47:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 15:47:14.903126 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae35a19-2434-415b-9a83-ad3c1a12a8a0 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:14.903414 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae35a19-2434-415b-9a83-ad3c1a12a8a0 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:14.903680 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae35a19-2434-415b-9a83-ad3c1a12a8a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:14.922504 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-4ae35a19-2434-415b-9a83-ad3c1a12a8a0 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:47:14.923704 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-4ae35a19-2434-415b-9a83-ad3c1a12a8a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:47:14.944553 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e6986b-298c-4d48-a55c-a192c113297d None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:14.964996 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:47:14.984427 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113a19d4-58cb-4ff1-acef-a1e1d88e3c73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:14.985351 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113a19d4-58cb-4ff1-acef-a1e1d88e3c73 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:14.985351 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113a19d4-58cb-4ff1-acef-a1e1d88e3c73 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:14.985673 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113a19d4-58cb-4ff1-acef-a1e1d88e3c73 None admin] PATH_INFO: `/v3/domains` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:14.986323 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113a19d4-58cb-4ff1-acef-a1e1d88e3c73 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81512) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:47:14.986674 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113a19d4-58cb-4ff1-acef-a1e1d88e3c73 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:14.988063 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113a19d4-58cb-4ff1-acef-a1e1d88e3c73 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:15.001359 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 454/898] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Aug 30 15:47:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:47:15.006873 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113a19d4-58cb-4ff1-acef-a1e1d88e3c73 None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:15.015155 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 445/899] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 30 15:47:14 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 30 15:47:15.110297 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113a19d4-58cb-4ff1-acef-a1e1d88e3c73 None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:15.146089 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45817e82-2675-4de7-9d6b-496b0ccee204 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:15.146959 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45817e82-2675-4de7-9d6b-496b0ccee204 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:15.147385 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45817e82-2675-4de7-9d6b-496b0ccee204 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:15.147798 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45817e82-2675-4de7-9d6b-496b0ccee204 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:15.155415 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45817e82-2675-4de7-9d6b-496b0ccee204 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:15.157064 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45817e82-2675-4de7-9d6b-496b0ccee204 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:15.169237 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application [None req-45817e82-2675-4de7-9d6b-496b0ccee204 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 30 15:47:15.169237 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:47:15.169237 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:47:15.169237 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:47:15.169237 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:47:15.169237 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:47:15.169237 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:47:15.169237 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:47:15.169237 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:47:15.169237 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:47:15.169237 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:47:15.169237 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:47:15.169237 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 15:47:15.169237 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 15:47:15.169237 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 15:47:15.169237 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 15:47:15.170537 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:47:15.170537 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:47:15.170537 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 15:47:15.170537 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 15:47:15.170537 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 15:47:15.170537 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 15:47:15.170537 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:47:15.170537 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:47:15.170537 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:47:15.170537 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:47:15.170537 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:47:15.170537 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application with Lock( Aug 30 15:47:15.170537 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:47:15.170537 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:47:15.170537 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:47:15.170537 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:47:15.170537 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:47:15.170537 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:47:15.171728 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:47:15.171728 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:47:15.171728 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 15:47:15.171728 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 15:47:15.171728 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 15:47:15.171728 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 15:47:15.171728 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 15:47:15.171728 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 15:47:15.171728 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 30 15:47:15.171728 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Aug 30 15:47:15.174014 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 455/900] 10.222.0.20 () {56 vars in 1117 bytes} [Tue Aug 30 15:47:15 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 15:47:15.182471 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae35a19-2434-415b-9a83-ad3c1a12a8a0 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:15.232819 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:47:15.278669 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45817e82-2675-4de7-9d6b-496b0ccee204 None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:15.289069 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae35a19-2434-415b-9a83-ad3c1a12a8a0 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:15.320131 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:47:15.336010 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cddb02d-c0e8-44c5-99b2-8596bf1e3232 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:15.337142 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cddb02d-c0e8-44c5-99b2-8596bf1e3232 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:15.337628 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cddb02d-c0e8-44c5-99b2-8596bf1e3232 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:15.338205 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cddb02d-c0e8-44c5-99b2-8596bf1e3232 None admin] PATH_INFO: `/v3/users` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:15.339414 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cddb02d-c0e8-44c5-99b2-8596bf1e3232 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:47:15.340134 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cddb02d-c0e8-44c5-99b2-8596bf1e3232 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:15.342257 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cddb02d-c0e8-44c5-99b2-8596bf1e3232 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:15.370744 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 446/901] 10.222.0.20 () {56 vars in 1149 bytes} [Tue Aug 30 15:47:15 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:47:15.380957 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cddb02d-c0e8-44c5-99b2-8596bf1e3232 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:15.389747 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45817e82-2675-4de7-9d6b-496b0ccee204 None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:15.424748 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc3583e-2528-4177-87b4-eecf4bb1175c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:15.425844 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc3583e-2528-4177-87b4-eecf4bb1175c None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:15.426496 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc3583e-2528-4177-87b4-eecf4bb1175c None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:15.426963 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc3583e-2528-4177-87b4-eecf4bb1175c None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:15.427743 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc3583e-2528-4177-87b4-eecf4bb1175c None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:15.429451 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc3583e-2528-4177-87b4-eecf4bb1175c None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:15.438879 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application [None req-9fc3583e-2528-4177-87b4-eecf4bb1175c None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 30 15:47:15.438879 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:47:15.438879 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 15:47:15.438879 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 15:47:15.438879 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 15:47:15.438879 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 15:47:15.438879 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 15:47:15.438879 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 15:47:15.438879 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 30 15:47:15.438879 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Aug 30 15:47:15.438879 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 15:47:15.438879 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Aug 30 15:47:15.438879 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:47:15.438879 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:47:15.438879 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:47:15.438879 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:47:15.439815 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:47:15.439815 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:47:15.439815 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:47:15.439815 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:47:15.439815 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:47:15.439815 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:47:15.439815 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:47:15.439815 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 15:47:15.439815 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 15:47:15.439815 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 15:47:15.439815 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 15:47:15.439815 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:47:15.439815 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:47:15.439815 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:47:15.439815 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:47:15.439815 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:47:15.439815 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:47:15.440713 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:47:15.440713 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application with Lock( Aug 30 15:47:15.440713 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:47:15.440713 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:47:15.440713 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:47:15.440713 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:47:15.440713 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:47:15.440713 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:47:15.440713 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:47:15.440713 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:47:15.440713 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 15:47:15.440713 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 15:47:15.440713 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 30 15:47:15.440713 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Aug 30 15:47:15.448622 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 456/902] 10.222.0.20 () {56 vars in 1132 bytes} [Tue Aug 30 15:47:15 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 15:47:15.449928 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc3583e-2528-4177-87b4-eecf4bb1175c None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:15.486285 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cddb02d-c0e8-44c5-99b2-8596bf1e3232 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:15.493759 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:47:15.522937 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bb607b-8ec0-4681-90b0-0065758d5f51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:15.523552 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bb607b-8ec0-4681-90b0-0065758d5f51 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:15.523828 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bb607b-8ec0-4681-90b0-0065758d5f51 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:15.524096 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bb607b-8ec0-4681-90b0-0065758d5f51 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:15.574139 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc3583e-2528-4177-87b4-eecf4bb1175c None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:15.576694 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:47:15.596680 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bb607b-8ec0-4681-90b0-0065758d5f51 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:15.598891 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bb607b-8ec0-4681-90b0-0065758d5f51 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:15.624334 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b31afbb-e0f1-452e-b1fb-edc1d5ebb452 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:15.625642 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b31afbb-e0f1-452e-b1fb-edc1d5ebb452 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:15.626130 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b31afbb-e0f1-452e-b1fb-edc1d5ebb452 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:15.626598 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b31afbb-e0f1-452e-b1fb-edc1d5ebb452 None admin] PATH_INFO: `/v3/services` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:15.627474 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b31afbb-e0f1-452e-b1fb-edc1d5ebb452 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81512) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:47:15.627994 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b31afbb-e0f1-452e-b1fb-edc1d5ebb452 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:15.629636 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b31afbb-e0f1-452e-b1fb-edc1d5ebb452 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:15.642274 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 457/903] 10.222.0.20 () {56 vars in 1126 bytes} [Tue Aug 30 15:47:15 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:47:15.643918 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b31afbb-e0f1-452e-b1fb-edc1d5ebb452 None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:15.713884 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 447/904] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Aug 30 15:47:15 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 30 15:47:15.719223 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bb607b-8ec0-4681-90b0-0065758d5f51 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:15.727539 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:47:15.788439 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b31afbb-e0f1-452e-b1fb-edc1d5ebb452 None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:15.823062 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be046d81-5e06-43bd-b37e-f765a47ca954 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:15.823662 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be046d81-5e06-43bd-b37e-f765a47ca954 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:15.823936 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be046d81-5e06-43bd-b37e-f765a47ca954 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:15.824198 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be046d81-5e06-43bd-b37e-f765a47ca954 None admin] PATH_INFO: `/v3/services` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:15.824927 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be046d81-5e06-43bd-b37e-f765a47ca954 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81512) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:47:15.825234 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be046d81-5e06-43bd-b37e-f765a47ca954 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:15.826838 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be046d81-5e06-43bd-b37e-f765a47ca954 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:15.836956 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 458/905] 10.222.0.20 () {56 vars in 1126 bytes} [Tue Aug 30 15:47:15 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:47:15.836956 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090764d7-3d90-45d8-8c70-0c33303f6489 None None] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:15.836956 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090764d7-3d90-45d8-8c70-0c33303f6489 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:15.836956 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090764d7-3d90-45d8-8c70-0c33303f6489 None None] PATH_INFO: `/` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:15.837392 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bb607b-8ec0-4681-90b0-0065758d5f51 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:15.839710 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 459/906] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 30 15:47:15 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 15:47:15.839831 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090764d7-3d90-45d8-8c70-0c33303f6489 None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:15.870508 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23da7558-1e99-4a91-8b0c-9d8d0584cc35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:15.871145 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23da7558-1e99-4a91-8b0c-9d8d0584cc35 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:15.871417 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23da7558-1e99-4a91-8b0c-9d8d0584cc35 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:15.871515 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:47:15.871678 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23da7558-1e99-4a91-8b0c-9d8d0584cc35 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:15.883101 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23da7558-1e99-4a91-8b0c-9d8d0584cc35 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:15.884608 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23da7558-1e99-4a91-8b0c-9d8d0584cc35 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:15.902878 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-23da7558-1e99-4a91-8b0c-9d8d0584cc35 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 15:47:15.902878 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:47:15.902878 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:47:15.902878 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:47:15.902878 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:47:15.902878 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:47:15.902878 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:47:15.902878 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:47:15.902878 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:47:15.902878 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:47:15.902878 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:47:15.902878 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:47:15.902878 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 15:47:15.902878 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 15:47:15.902878 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 15:47:15.902878 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 15:47:15.903946 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:47:15.903946 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:47:15.903946 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:47:15.903946 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:47:15.903946 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:47:15.903946 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:47:15.903946 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:47:15.903946 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Aug 30 15:47:15.903946 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:47:15.903946 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:47:15.903946 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:47:15.903946 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:47:15.903946 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:47:15.903946 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:47:15.903946 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:47:15.903946 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:47:15.903946 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 15:47:15.904901 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 15:47:15.904901 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 15:47:15.904901 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Aug 30 15:47:15.904901 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 448/907] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 30 15:47:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 15:47:15.905667 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ba4c9d-a1e4-499f-b281-fe8d53098d02 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:15.905939 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ba4c9d-a1e4-499f-b281-fe8d53098d02 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:15.906211 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ba4c9d-a1e4-499f-b281-fe8d53098d02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:15.925014 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-e9ba4c9d-a1e4-499f-b281-fe8d53098d02 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:47:15.927173 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-e9ba4c9d-a1e4-499f-b281-fe8d53098d02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:47:15.965909 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090764d7-3d90-45d8-8c70-0c33303f6489 None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:15.983911 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:47:16.000241 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 449/908] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 30 15:47:15 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 15:47:16.001254 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ba4c9d-a1e4-499f-b281-fe8d53098d02 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:16.006019 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36237b1b-ae3a-45b1-beb7-74ab1756fa90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:16.006918 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36237b1b-ae3a-45b1-beb7-74ab1756fa90 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:16.007453 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36237b1b-ae3a-45b1-beb7-74ab1756fa90 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:16.007910 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36237b1b-ae3a-45b1-beb7-74ab1756fa90 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:16.008835 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36237b1b-ae3a-45b1-beb7-74ab1756fa90 None admin] RBAC: Adding query filter params (interface=public, service_id=ca4dc4fcf64a449f9a9c9eebae7f7e6f, region_id=RegionOne) {{(pid=81512) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:47:16.009374 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36237b1b-ae3a-45b1-beb7-74ab1756fa90 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:16.011025 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36237b1b-ae3a-45b1-beb7-74ab1756fa90 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:16.016660 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 460/909] 10.222.0.20 () {56 vars in 1251 bytes} [Tue Aug 30 15:47:15 2022] GET /identity/v3/endpoints?service_id=ca4dc4fcf64a449f9a9c9eebae7f7e6f&interface=public®ion_id=RegionOne => generated 194 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:47:16.022838 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36237b1b-ae3a-45b1-beb7-74ab1756fa90 None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:16.113269 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ba4c9d-a1e4-499f-b281-fe8d53098d02 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:16.136334 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:47:16.146036 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36237b1b-ae3a-45b1-beb7-74ab1756fa90 None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:16.152723 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567838d4-7c8e-4587-9415-362a6530dab4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:16.153388 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567838d4-7c8e-4587-9415-362a6530dab4 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:16.153668 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567838d4-7c8e-4587-9415-362a6530dab4 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:16.153936 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567838d4-7c8e-4587-9415-362a6530dab4 None admin] PATH_INFO: `/v3/domains` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:16.154658 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567838d4-7c8e-4587-9415-362a6530dab4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:47:16.154956 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567838d4-7c8e-4587-9415-362a6530dab4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:16.156457 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567838d4-7c8e-4587-9415-362a6530dab4 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:16.171939 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 450/910] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Aug 30 15:47:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:47:16.177809 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567838d4-7c8e-4587-9415-362a6530dab4 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:16.200326 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144d0443-67e1-4e25-9748-adb9233011e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:16.200955 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144d0443-67e1-4e25-9748-adb9233011e1 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:16.201376 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144d0443-67e1-4e25-9748-adb9233011e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:16.201651 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144d0443-67e1-4e25-9748-adb9233011e1 None admin] PATH_INFO: `/v3/services` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:16.202624 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144d0443-67e1-4e25-9748-adb9233011e1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:16.204347 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144d0443-67e1-4e25-9748-adb9233011e1 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:16.212110 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 461/911] 10.222.0.20 () {56 vars in 1087 bytes} [Tue Aug 30 15:47:16 2022] GET /identity/v3/services => generated 1497 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 15:47:16.246708 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:47:16.295231 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144d0443-67e1-4e25-9748-adb9233011e1 None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:16.312249 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567838d4-7c8e-4587-9415-362a6530dab4 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:16.362004 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab160bb8-f670-4783-8c60-151e7b82ba65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:16.362954 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab160bb8-f670-4783-8c60-151e7b82ba65 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:16.363461 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab160bb8-f670-4783-8c60-151e7b82ba65 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:16.364009 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab160bb8-f670-4783-8c60-151e7b82ba65 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:16.372330 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab160bb8-f670-4783-8c60-151e7b82ba65 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:16.374254 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab160bb8-f670-4783-8c60-151e7b82ba65 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:16.390236 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-ab160bb8-f670-4783-8c60-151e7b82ba65 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 15:47:16.390236 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:47:16.390236 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:47:16.390236 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:47:16.390236 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:47:16.390236 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:47:16.390236 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:47:16.390236 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:47:16.390236 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:47:16.390236 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:47:16.390236 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:47:16.390236 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:47:16.390236 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 15:47:16.390236 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 15:47:16.390236 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 15:47:16.391349 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 15:47:16.391349 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:47:16.391349 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:47:16.391349 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:47:16.391349 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:47:16.391349 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:47:16.391349 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:47:16.391349 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:47:16.391349 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Aug 30 15:47:16.391349 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:47:16.391349 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:47:16.391349 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:47:16.391349 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:47:16.391349 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:47:16.391349 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:47:16.391349 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:47:16.391349 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:47:16.392469 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 15:47:16.392469 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 15:47:16.392469 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 15:47:16.392469 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 15:47:16.392469 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 15:47:16.392469 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 15:47:16.392469 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 15:47:16.392469 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Aug 30 15:47:16.392938 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 451/912] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Aug 30 15:47:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 15:47:16.400123 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab160bb8-f670-4783-8c60-151e7b82ba65 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:16.412364 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144d0443-67e1-4e25-9748-adb9233011e1 None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:16.456751 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e2a8c0-f1d3-449a-9678-4f3bfe571e0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:16.457799 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e2a8c0-f1d3-449a-9678-4f3bfe571e0b None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:16.458082 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e2a8c0-f1d3-449a-9678-4f3bfe571e0b None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:16.458355 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e2a8c0-f1d3-449a-9678-4f3bfe571e0b None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:16.458951 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e2a8c0-f1d3-449a-9678-4f3bfe571e0b None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:16.460844 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e2a8c0-f1d3-449a-9678-4f3bfe571e0b None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:16.473652 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application [None req-a2e2a8c0-f1d3-449a-9678-4f3bfe571e0b None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 30 15:47:16.473652 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:47:16.473652 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 15:47:16.473652 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 15:47:16.473652 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 15:47:16.473652 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 15:47:16.473652 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 15:47:16.473652 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 15:47:16.473652 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 30 15:47:16.473652 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Aug 30 15:47:16.473652 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 15:47:16.473652 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Aug 30 15:47:16.473652 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:47:16.473652 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:47:16.473652 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:47:16.473652 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:47:16.473652 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:47:16.474673 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:47:16.474673 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:47:16.474673 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:47:16.474673 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:47:16.474673 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:47:16.474673 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:47:16.474673 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 15:47:16.474673 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 15:47:16.474673 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 15:47:16.474673 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 15:47:16.474673 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:47:16.474673 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:47:16.474673 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:47:16.474673 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:47:16.474673 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:47:16.474673 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:47:16.475533 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:47:16.475533 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application with Lock( Aug 30 15:47:16.475533 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:47:16.475533 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:47:16.475533 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:47:16.475533 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:47:16.475533 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:47:16.475533 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:47:16.475533 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:47:16.475533 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:47:16.475533 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 15:47:16.475533 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 15:47:16.475533 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 30 15:47:16.475533 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Aug 30 15:47:16.476281 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 462/913] 10.222.0.20 () {56 vars in 1114 bytes} [Tue Aug 30 15:47:16 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 15:47:16.481746 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e2a8c0-f1d3-449a-9678-4f3bfe571e0b None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:16.510340 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:47:16.514116 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab160bb8-f670-4783-8c60-151e7b82ba65 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:16.523843 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:47:16.550266 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79b920b-e144-4faf-8672-40180c198c2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:16.551023 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79b920b-e144-4faf-8672-40180c198c2d None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:16.551390 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79b920b-e144-4faf-8672-40180c198c2d None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:16.551732 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79b920b-e144-4faf-8672-40180c198c2d None admin] PATH_INFO: `/v3/projects` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:16.552493 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79b920b-e144-4faf-8672-40180c198c2d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:47:16.552874 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79b920b-e144-4faf-8672-40180c198c2d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:16.554609 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79b920b-e144-4faf-8672-40180c198c2d None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:16.567785 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 452/914] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Aug 30 15:47:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:47:16.573922 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79b920b-e144-4faf-8672-40180c198c2d None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:16.595056 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e2a8c0-f1d3-449a-9678-4f3bfe571e0b None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:16.632808 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edad5905-718a-447f-b07d-b14d63f9aeb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:16.633464 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edad5905-718a-447f-b07d-b14d63f9aeb8 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:16.633850 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edad5905-718a-447f-b07d-b14d63f9aeb8 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:16.634128 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edad5905-718a-447f-b07d-b14d63f9aeb8 None admin] PATH_INFO: `/v3/services` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:16.634774 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edad5905-718a-447f-b07d-b14d63f9aeb8 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81512) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:47:16.635077 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edad5905-718a-447f-b07d-b14d63f9aeb8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:16.636561 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edad5905-718a-447f-b07d-b14d63f9aeb8 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:16.647121 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 463/915] 10.222.0.20 () {56 vars in 1114 bytes} [Tue Aug 30 15:47:16 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:47:16.648345 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edad5905-718a-447f-b07d-b14d63f9aeb8 None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:16.679897 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79b920b-e144-4faf-8672-40180c198c2d None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:16.718946 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca970167-edf9-45c8-9c91-fe52d6cd38a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:16.720215 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca970167-edf9-45c8-9c91-fe52d6cd38a5 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:16.721210 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca970167-edf9-45c8-9c91-fe52d6cd38a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:16.721706 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca970167-edf9-45c8-9c91-fe52d6cd38a5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:16.722951 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca970167-edf9-45c8-9c91-fe52d6cd38a5 None admin] RBAC: Adding query filter params (role.id=a89e0730512d449ca32479022aaf58d1, scope.project.id=854fe71b49224dff9a4b1766a20ab266, user.id=07130d598131443486c2bc1c6673c45b) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:47:16.723665 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca970167-edf9-45c8-9c91-fe52d6cd38a5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:16.726312 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca970167-edf9-45c8-9c91-fe52d6cd38a5 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:16.734423 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 453/916] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Aug 30 15:47:16 2022] GET /identity/v3/role_assignments?user.id=07130d598131443486c2bc1c6673c45b&scope.project.id=854fe71b49224dff9a4b1766a20ab266&role.id=a89e0730512d449ca32479022aaf58d1 => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 15:47:16.736606 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43ef3b6-3f04-4d5b-a16e-f07a253e2b6a None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:16.736896 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43ef3b6-3f04-4d5b-a16e-f07a253e2b6a None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:16.737388 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43ef3b6-3f04-4d5b-a16e-f07a253e2b6a None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:16.738854 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 454/917] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 30 15:47:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 15:47:16.744024 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:47:16.746199 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2daa9d-91a1-458e-b818-65c3596c5d02 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:16.746651 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2daa9d-91a1-458e-b818-65c3596c5d02 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:16.747070 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2daa9d-91a1-458e-b818-65c3596c5d02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:16.764739 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edad5905-718a-447f-b07d-b14d63f9aeb8 None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:16.766823 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-3a2daa9d-91a1-458e-b818-65c3596c5d02 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:47:16.768119 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-3a2daa9d-91a1-458e-b818-65c3596c5d02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:47:16.798680 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:47:16.813705 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7f59e8-cde3-4792-a43c-d444a1ea52e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:16.814590 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7f59e8-cde3-4792-a43c-d444a1ea52e2 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:16.815031 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7f59e8-cde3-4792-a43c-d444a1ea52e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:16.815435 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7f59e8-cde3-4792-a43c-d444a1ea52e2 None admin] PATH_INFO: `/v3/services` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:16.816993 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7f59e8-cde3-4792-a43c-d444a1ea52e2 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81512) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:47:16.817487 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7f59e8-cde3-4792-a43c-d444a1ea52e2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:16.819607 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7f59e8-cde3-4792-a43c-d444a1ea52e2 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:16.827791 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 464/918] 10.222.0.20 () {56 vars in 1114 bytes} [Tue Aug 30 15:47:16 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:47:16.834334 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7f59e8-cde3-4792-a43c-d444a1ea52e2 None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:16.848231 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 455/919] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 30 15:47:16 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 15:47:16.942019 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7f59e8-cde3-4792-a43c-d444a1ea52e2 None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:16.977739 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4248310e-8137-4271-bb78-a65f28488ae7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:16.978773 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4248310e-8137-4271-bb78-a65f28488ae7 None admin] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:16.979217 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4248310e-8137-4271-bb78-a65f28488ae7 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:16.979658 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4248310e-8137-4271-bb78-a65f28488ae7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:16.980482 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4248310e-8137-4271-bb78-a65f28488ae7 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:16.982064 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4248310e-8137-4271-bb78-a65f28488ae7 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:17.006845 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 465/920] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Aug 30 15:47:16 2022] POST /identity/v3/endpoints => generated 346 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 15:47:17.125948 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2daa9d-91a1-458e-b818-65c3596c5d02 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:17.230831 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2daa9d-91a1-458e-b818-65c3596c5d02 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:17.274756 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1adc2f-f186-4cf0-b21d-463be2b87e81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:17.275387 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1adc2f-f186-4cf0-b21d-463be2b87e81 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:17.275657 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1adc2f-f186-4cf0-b21d-463be2b87e81 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:17.275934 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1adc2f-f186-4cf0-b21d-463be2b87e81 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:17.276513 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1adc2f-f186-4cf0-b21d-463be2b87e81 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:17.277935 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1adc2f-f186-4cf0-b21d-463be2b87e81 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:17.287129 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-cc1adc2f-f186-4cf0-b21d-463be2b87e81 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 30 15:47:17.287129 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:47:17.287129 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 15:47:17.287129 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 15:47:17.287129 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 15:47:17.287129 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 15:47:17.287129 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 15:47:17.287129 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 15:47:17.287129 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 30 15:47:17.287129 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Aug 30 15:47:17.287129 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 15:47:17.287129 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Aug 30 15:47:17.287129 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:47:17.287129 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:47:17.287129 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:47:17.287129 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:47:17.287129 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:47:17.288332 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:47:17.288332 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:47:17.288332 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:47:17.288332 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:47:17.288332 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:47:17.288332 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:47:17.288332 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 15:47:17.288332 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 15:47:17.288332 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 15:47:17.288332 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 15:47:17.288332 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:47:17.288332 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:47:17.288332 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:47:17.288332 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:47:17.288332 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:47:17.288332 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:47:17.289461 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:47:17.289461 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Aug 30 15:47:17.289461 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:47:17.289461 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:47:17.289461 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:47:17.289461 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:47:17.289461 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:47:17.289461 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:47:17.289461 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:47:17.289461 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:47:17.289461 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 15:47:17.289461 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 15:47:17.289461 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 30 15:47:17.289461 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Aug 30 15:47:17.289461 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 456/921] 10.222.0.20 () {56 vars in 1126 bytes} [Tue Aug 30 15:47:17 2022] GET /identity/v3/services/object-store => generated 93 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 15:47:17.294353 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4248310e-8137-4271-bb78-a65f28488ae7 None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:17.308088 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:47:17.404331 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4248310e-8137-4271-bb78-a65f28488ae7 None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:17.438353 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf42bce-c9ab-4339-bf56-f15706a5bcb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:17.439317 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf42bce-c9ab-4339-bf56-f15706a5bcb8 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:17.439815 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf42bce-c9ab-4339-bf56-f15706a5bcb8 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:17.440298 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf42bce-c9ab-4339-bf56-f15706a5bcb8 None admin] PATH_INFO: `/v3/services` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:17.441117 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf42bce-c9ab-4339-bf56-f15706a5bcb8 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81512) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:47:17.441653 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf42bce-c9ab-4339-bf56-f15706a5bcb8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:17.443283 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf42bce-c9ab-4339-bf56-f15706a5bcb8 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:17.449690 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 466/922] 10.222.0.20 () {56 vars in 1122 bytes} [Tue Aug 30 15:47:17 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:47:17.458095 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1adc2f-f186-4cf0-b21d-463be2b87e81 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:17.483889 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:47:17.566130 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1adc2f-f186-4cf0-b21d-463be2b87e81 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:17.599993 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34926cb2-a096-40d6-a7cf-88dfe8a13bcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:17.600610 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34926cb2-a096-40d6-a7cf-88dfe8a13bcf None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:17.600886 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34926cb2-a096-40d6-a7cf-88dfe8a13bcf None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:17.601155 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34926cb2-a096-40d6-a7cf-88dfe8a13bcf None admin] PATH_INFO: `/v3/services` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:17.601820 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34926cb2-a096-40d6-a7cf-88dfe8a13bcf None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:47:17.602117 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34926cb2-a096-40d6-a7cf-88dfe8a13bcf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:17.603497 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34926cb2-a096-40d6-a7cf-88dfe8a13bcf None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:17.608793 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 457/923] 10.222.0.20 () {56 vars in 1122 bytes} [Tue Aug 30 15:47:17 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:47:17.614472 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf42bce-c9ab-4339-bf56-f15706a5bcb8 None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:17.687210 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:47:17.723907 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf42bce-c9ab-4339-bf56-f15706a5bcb8 None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:17.758866 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f09909-ea9a-4bac-ad32-d568841a18bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:17.759813 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f09909-ea9a-4bac-ad32-d568841a18bb None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:17.760258 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f09909-ea9a-4bac-ad32-d568841a18bb None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:17.760688 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f09909-ea9a-4bac-ad32-d568841a18bb None admin] PATH_INFO: `/v3/endpoints` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:17.761584 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f09909-ea9a-4bac-ad32-d568841a18bb None admin] RBAC: Adding query filter params (interface=public, service_id=cbb1b96d587744728e67a883846db270, region_id=RegionOne) {{(pid=81512) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:47:17.762056 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f09909-ea9a-4bac-ad32-d568841a18bb None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:17.763665 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f09909-ea9a-4bac-ad32-d568841a18bb None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:17.769599 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 467/924] 10.222.0.20 () {56 vars in 1251 bytes} [Tue Aug 30 15:47:17 2022] GET /identity/v3/endpoints?service_id=cbb1b96d587744728e67a883846db270&interface=public®ion_id=RegionOne => generated 194 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:47:17.774912 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34926cb2-a096-40d6-a7cf-88dfe8a13bcf None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:17.867323 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:47:17.889267 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34926cb2-a096-40d6-a7cf-88dfe8a13bcf None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:17.921061 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354da979-15f2-4099-98ec-b5013a101e18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:17.921704 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354da979-15f2-4099-98ec-b5013a101e18 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:17.921972 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354da979-15f2-4099-98ec-b5013a101e18 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:17.922244 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354da979-15f2-4099-98ec-b5013a101e18 None admin] PATH_INFO: `/v3/services` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:17.922858 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354da979-15f2-4099-98ec-b5013a101e18 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:17.924234 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354da979-15f2-4099-98ec-b5013a101e18 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:17.930707 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 458/925] 10.222.0.20 () {56 vars in 1087 bytes} [Tue Aug 30 15:47:17 2022] GET /identity/v3/services => generated 1497 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 15:47:18.287451 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f7ed04-bdd7-4110-bd49-dd2925173e46 None None] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:18.287867 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f7ed04-bdd7-4110-bd49-dd2925173e46 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:18.288045 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f7ed04-bdd7-4110-bd49-dd2925173e46 None None] PATH_INFO: `/` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:18.289640 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 468/926] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 30 15:47:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 15:47:18.295146 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e3f585-dcc6-4609-a8fa-598084e7fd18 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:18.295598 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e3f585-dcc6-4609-a8fa-598084e7fd18 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:18.295766 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e3f585-dcc6-4609-a8fa-598084e7fd18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:18.314165 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-d5e3f585-dcc6-4609-a8fa-598084e7fd18 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:47:18.315361 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-d5e3f585-dcc6-4609-a8fa-598084e7fd18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:47:18.385346 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 459/927] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 30 15:47:18 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 30 15:47:18.655058 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f7ed04-bdd7-4110-bd49-dd2925173e46 None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:18.719263 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51af0eea-0d15-4db3-bffc-1cbf819af3d8 None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:18.719599 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51af0eea-0d15-4db3-bffc-1cbf819af3d8 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:18.720750 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51af0eea-0d15-4db3-bffc-1cbf819af3d8 None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:18.724972 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 460/928] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 30 15:47:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 15:47:18.726614 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983d7b95-50d3-445a-84c1-0cd9fd69abaa None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:18.726900 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983d7b95-50d3-445a-84c1-0cd9fd69abaa None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:18.727174 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983d7b95-50d3-445a-84c1-0cd9fd69abaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:18.750052 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-983d7b95-50d3-445a-84c1-0cd9fd69abaa None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:47:18.751245 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-983d7b95-50d3-445a-84c1-0cd9fd69abaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:47:18.768888 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f7ed04-bdd7-4110-bd49-dd2925173e46 None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:18.813255 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e372b8b3-5705-42a8-ab92-2dd04cfc62a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:18.814430 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e372b8b3-5705-42a8-ab92-2dd04cfc62a4 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:18.814928 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e372b8b3-5705-42a8-ab92-2dd04cfc62a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:18.815698 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e372b8b3-5705-42a8-ab92-2dd04cfc62a4 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:18.816797 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e372b8b3-5705-42a8-ab92-2dd04cfc62a4 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:18.819059 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e372b8b3-5705-42a8-ab92-2dd04cfc62a4 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:18.821321 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:47:18.829134 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application [None req-e372b8b3-5705-42a8-ab92-2dd04cfc62a4 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 30 15:47:18.829134 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:47:18.829134 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 15:47:18.829134 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 15:47:18.829134 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 15:47:18.829134 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 15:47:18.829134 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 15:47:18.829134 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 15:47:18.829134 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 30 15:47:18.829134 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Aug 30 15:47:18.829134 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 15:47:18.829134 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Aug 30 15:47:18.829134 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:47:18.829134 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:47:18.829134 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:47:18.829134 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:47:18.830103 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:47:18.830103 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:47:18.830103 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:47:18.830103 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:47:18.830103 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:47:18.830103 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:47:18.830103 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:47:18.830103 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 15:47:18.830103 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 15:47:18.830103 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 15:47:18.830103 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 15:47:18.830103 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:47:18.830103 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:47:18.830103 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:47:18.830103 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:47:18.830103 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:47:18.830103 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:47:18.830993 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:47:18.830993 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application with Lock( Aug 30 15:47:18.830993 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:47:18.830993 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:47:18.830993 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:47:18.830993 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:47:18.830993 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:47:18.830993 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:47:18.830993 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:47:18.830993 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:47:18.830993 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 15:47:18.830993 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 15:47:18.830993 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 30 15:47:18.830993 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Aug 30 15:47:18.831736 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 461/929] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 30 15:47:18 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 15:47:18.833746 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 469/930] 10.222.0.20 () {56 vars in 1132 bytes} [Tue Aug 30 15:47:18 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 15:47:18.839557 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983d7b95-50d3-445a-84c1-0cd9fd69abaa None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:18.852241 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:47:18.943313 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983d7b95-50d3-445a-84c1-0cd9fd69abaa None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:18.979463 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8134b8f-c544-4b2d-ba9b-722c2a95f2fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:18.980378 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8134b8f-c544-4b2d-ba9b-722c2a95f2fa None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:18.980821 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8134b8f-c544-4b2d-ba9b-722c2a95f2fa None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:18.981292 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8134b8f-c544-4b2d-ba9b-722c2a95f2fa None admin] PATH_INFO: `/v3/services` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:18.982108 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8134b8f-c544-4b2d-ba9b-722c2a95f2fa None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:47:18.982610 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8134b8f-c544-4b2d-ba9b-722c2a95f2fa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:18.984149 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8134b8f-c544-4b2d-ba9b-722c2a95f2fa None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:18.989621 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 462/931] 10.222.0.20 () {56 vars in 1126 bytes} [Tue Aug 30 15:47:18 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 15:47:18.998447 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e372b8b3-5705-42a8-ab92-2dd04cfc62a4 None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:19.050773 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78f02a0-e75d-4044-9f13-c38098cf58f0 None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:19.051759 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78f02a0-e75d-4044-9f13-c38098cf58f0 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:19.051759 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78f02a0-e75d-4044-9f13-c38098cf58f0 None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:19.053963 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 463/932] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 30 15:47:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 15:47:19.059255 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2c2658-bbb1-42df-9fef-77f0c3f7556d None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:19.059595 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2c2658-bbb1-42df-9fef-77f0c3f7556d None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:19.059864 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2c2658-bbb1-42df-9fef-77f0c3f7556d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:19.079056 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-8c2c2658-bbb1-42df-9fef-77f0c3f7556d None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:47:19.080390 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-8c2c2658-bbb1-42df-9fef-77f0c3f7556d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:47:19.118276 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e372b8b3-5705-42a8-ab92-2dd04cfc62a4 None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:19.153875 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 464/933] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 30 15:47:19 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 15:47:19.155006 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2c2658-bbb1-42df-9fef-77f0c3f7556d None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:19.159169 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:47:19.160394 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06750e8-270a-48ca-aa86-9af56fa39ede None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:19.161330 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06750e8-270a-48ca-aa86-9af56fa39ede None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:19.161819 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06750e8-270a-48ca-aa86-9af56fa39ede None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:19.162782 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06750e8-270a-48ca-aa86-9af56fa39ede None admin] PATH_INFO: `/v3/services` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:19.163794 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06750e8-270a-48ca-aa86-9af56fa39ede None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81512) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:47:19.164272 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06750e8-270a-48ca-aa86-9af56fa39ede None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:19.165859 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06750e8-270a-48ca-aa86-9af56fa39ede None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:19.171746 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 470/934] 10.222.0.20 () {56 vars in 1126 bytes} [Tue Aug 30 15:47:18 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:47:19.177797 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06750e8-270a-48ca-aa86-9af56fa39ede None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:19.252980 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:47:19.284077 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2c2658-bbb1-42df-9fef-77f0c3f7556d None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:19.295052 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06750e8-270a-48ca-aa86-9af56fa39ede None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:19.329195 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839d0335-0b17-4373-9bed-d84f100e1dc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:19.330605 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839d0335-0b17-4373-9bed-d84f100e1dc1 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:19.331130 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839d0335-0b17-4373-9bed-d84f100e1dc1 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:19.331543 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839d0335-0b17-4373-9bed-d84f100e1dc1 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:19.331817 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfb1f3d-f263-455b-93b5-9e75e933ea29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:19.332508 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfb1f3d-f263-455b-93b5-9e75e933ea29 None admin] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:19.332891 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfb1f3d-f263-455b-93b5-9e75e933ea29 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:19.333273 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfb1f3d-f263-455b-93b5-9e75e933ea29 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:19.334127 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfb1f3d-f263-455b-93b5-9e75e933ea29 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:19.335801 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfb1f3d-f263-455b-93b5-9e75e933ea29 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:19.344270 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839d0335-0b17-4373-9bed-d84f100e1dc1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 15:47:19.350830 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-839d0335-0b17-4373-9bed-d84f100e1dc1 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 30 15:47:19.350830 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:47:19.350830 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:47:19.350830 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:47:19.350830 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:47:19.350830 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:47:19.350830 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:47:19.350830 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:47:19.350830 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:47:19.350830 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:47:19.350830 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:47:19.350830 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:47:19.350830 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 15:47:19.350830 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 15:47:19.350830 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 15:47:19.350830 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 15:47:19.351951 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 15:47:19.351951 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 15:47:19.351951 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 15:47:19.351951 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 15:47:19.351951 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:47:19.351951 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:47:19.351951 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:47:19.351951 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:47:19.351951 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:47:19.351951 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:47:19.351951 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:47:19.351951 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Aug 30 15:47:19.351951 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:47:19.351951 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:47:19.351951 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:47:19.351951 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:47:19.353928 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:47:19.353928 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:47:19.353928 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:47:19.353928 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:47:19.353928 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 15:47:19.353928 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 15:47:19.353928 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 15:47:19.353928 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 15:47:19.353928 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 15:47:19.353928 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 15:47:19.353928 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 30 15:47:19.353928 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Aug 30 15:47:19.353928 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 465/935] 10.222.0.20 () {56 vars in 1120 bytes} [Tue Aug 30 15:47:19 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 15:47:19.358189 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839d0335-0b17-4373-9bed-d84f100e1dc1 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:19.367516 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 471/936] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Aug 30 15:47:19 2022] POST /identity/v3/endpoints => generated 347 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 15:47:19.416921 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfb1f3d-f263-455b-93b5-9e75e933ea29 None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:19.444812 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:47:19.493990 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839d0335-0b17-4373-9bed-d84f100e1dc1 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:19.544228 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04cd224-f595-4712-be9c-502000196656 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:19.544886 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04cd224-f595-4712-be9c-502000196656 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:19.545245 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04cd224-f595-4712-be9c-502000196656 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:19.545576 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04cd224-f595-4712-be9c-502000196656 None admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:19.546298 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04cd224-f595-4712-be9c-502000196656 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:47:19.546796 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04cd224-f595-4712-be9c-502000196656 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:19.548125 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04cd224-f595-4712-be9c-502000196656 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:19.554730 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfb1f3d-f263-455b-93b5-9e75e933ea29 None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:19.563818 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 466/937] 10.222.0.20 () {56 vars in 1115 bytes} [Tue Aug 30 15:47:19 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 15:47:19.570703 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04cd224-f595-4712-be9c-502000196656 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:19.580715 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:47:19.605589 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c12f547-88f5-4b5b-b66d-950089a52206 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:19.606213 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c12f547-88f5-4b5b-b66d-950089a52206 None admin] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:19.606490 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c12f547-88f5-4b5b-b66d-950089a52206 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:19.606757 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c12f547-88f5-4b5b-b66d-950089a52206 None admin] PATH_INFO: `/v3/projects` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:19.607632 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c12f547-88f5-4b5b-b66d-950089a52206 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:19.609753 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c12f547-88f5-4b5b-b66d-950089a52206 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:19.613558 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: WARNING keystone.server.flask.common [None req-7c12f547-88f5-4b5b-b66d-950089a52206 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 30 15:47:19.642868 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.sql.core [None req-7c12f547-88f5-4b5b-b66d-950089a52206 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 30 15:47:19.642868 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 30 15:47:19.642868 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [parameters: {'id': '3892712072884b558d3ed972a01a2e62', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 30 15:47:19.642868 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81512) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 30 15:47:19.648319 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application [None req-7c12f547-88f5-4b5b-b66d-950089a52206 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 3892712072884b558d3ed972a01a2e62.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 3892712072884b558d3ed972a01a2e62. Aug 30 15:47:19.648319 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:47:19.648319 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 30 15:47:19.648319 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 30 15:47:19.648319 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 30 15:47:19.648319 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 30 15:47:19.648319 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 30 15:47:19.648319 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application result = self._query(query) Aug 30 15:47:19.648319 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 30 15:47:19.648319 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application conn.query(q) Aug 30 15:47:19.648319 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 30 15:47:19.648319 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 30 15:47:19.648319 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 30 15:47:19.649147 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application result.read() Aug 30 15:47:19.649147 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 30 15:47:19.649147 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 30 15:47:19.649147 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 30 15:47:19.649147 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 30 15:47:19.649147 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 30 15:47:19.649147 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 30 15:47:19.649147 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 30 15:47:19.649147 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 30 15:47:19.649147 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 30 15:47:19.649147 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Aug 30 15:47:19.649147 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 30 15:47:19.649147 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Aug 30 15:47:19.649147 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:47:19.649147 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 30 15:47:19.649147 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 30 15:47:19.649147 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Aug 30 15:47:19.649147 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return project_ref.to_dict() Aug 30 15:47:19.650201 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 30 15:47:19.650201 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application next(self.gen) Aug 30 15:47:19.650201 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 30 15:47:19.650201 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application yield resource Aug 30 15:47:19.650201 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 30 15:47:19.650201 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application next(self.gen) Aug 30 15:47:19.650201 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 30 15:47:19.650201 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application self.session.rollback() Aug 30 15:47:19.650201 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 30 15:47:19.650201 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application self.force_reraise() Aug 30 15:47:19.650201 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 30 15:47:19.650201 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application raise self.value Aug 30 15:47:19.650201 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 30 15:47:19.650201 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 30 15:47:19.650201 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 30 15:47:19.650201 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application session.commit() Aug 30 15:47:19.650201 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 30 15:47:19.651070 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 30 15:47:19.651070 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 30 15:47:19.651070 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application self._prepare_impl() Aug 30 15:47:19.651070 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 30 15:47:19.651070 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application self.session.flush() Aug 30 15:47:19.651070 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 30 15:47:19.651070 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application self._flush(objects) Aug 30 15:47:19.651070 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 30 15:47:19.651070 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 30 15:47:19.651070 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 30 15:47:19.651070 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application compat.raise_( Aug 30 15:47:19.651070 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 30 15:47:19.651070 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application raise exception Aug 30 15:47:19.651070 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 30 15:47:19.651070 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application flush_context.execute() Aug 30 15:47:19.651070 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 30 15:47:19.651070 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application rec.execute(self) Aug 30 15:47:19.651967 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 30 15:47:19.651967 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 30 15:47:19.651967 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 30 15:47:19.651967 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 30 15:47:19.651967 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 30 15:47:19.651967 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 30 15:47:19.651967 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 30 15:47:19.651967 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 30 15:47:19.651967 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 30 15:47:19.651967 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 30 15:47:19.651967 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 30 15:47:19.651967 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 30 15:47:19.651967 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 30 15:47:19.651967 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 30 15:47:19.651967 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 30 15:47:19.652765 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 30 15:47:19.652765 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 30 15:47:19.652765 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application raise exception Aug 30 15:47:19.652765 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 30 15:47:19.652765 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 30 15:47:19.652765 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 30 15:47:19.652765 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 30 15:47:19.652765 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 30 15:47:19.652765 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application result = self._query(query) Aug 30 15:47:19.652765 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 30 15:47:19.652765 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application conn.query(q) Aug 30 15:47:19.652765 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 30 15:47:19.652765 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 30 15:47:19.652765 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 30 15:47:19.652765 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application result.read() Aug 30 15:47:19.652765 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 30 15:47:19.652765 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 30 15:47:19.653672 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 30 15:47:19.653672 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 30 15:47:19.653672 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 30 15:47:19.653672 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 30 15:47:19.653672 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 30 15:47:19.653672 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 30 15:47:19.653672 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 30 15:47:19.653672 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 30 15:47:19.653672 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application [parameters: {'id': '3892712072884b558d3ed972a01a2e62', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 30 15:47:19.653672 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 30 15:47:19.653672 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Aug 30 15:47:19.653672 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 15:47:19.653672 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Aug 30 15:47:19.653672 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:47:19.653672 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Aug 30 15:47:19.654974 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Aug 30 15:47:19.654974 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 30 15:47:19.654974 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 30 15:47:19.654974 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Aug 30 15:47:19.654974 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Aug 30 15:47:19.654974 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 15:47:19.654974 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Aug 30 15:47:19.654974 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:47:19.654974 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:47:19.654974 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:47:19.654974 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:47:19.654974 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:47:19.654974 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:47:19.654974 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:47:19.654974 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:47:19.654974 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:47:19.654974 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:47:19.655918 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:47:19.655918 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Aug 30 15:47:19.655918 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Aug 30 15:47:19.655918 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:47:19.655918 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:47:19.655918 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Aug 30 15:47:19.655918 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application raise exception.Conflict( Aug 30 15:47:19.655918 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 3892712072884b558d3ed972a01a2e62. Aug 30 15:47:19.655918 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Aug 30 15:47:19.655918 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 472/938] 10.222.0.20 () {60 vars in 1140 bytes} [Tue Aug 30 15:47:19 2022] POST /identity/v3/projects => generated 262 bytes in 233 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Aug 30 15:47:19.655918 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c12f547-88f5-4b5b-b66d-950089a52206 None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:19.695776 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04cd224-f595-4712-be9c-502000196656 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:19.721054 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:47:19.734154 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb405cb-267c-4ea7-9afb-8b75488b1b43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:19.735294 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb405cb-267c-4ea7-9afb-8b75488b1b43 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:19.735748 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb405cb-267c-4ea7-9afb-8b75488b1b43 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:19.736196 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb405cb-267c-4ea7-9afb-8b75488b1b43 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:19.749631 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb405cb-267c-4ea7-9afb-8b75488b1b43 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:19.751414 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb405cb-267c-4ea7-9afb-8b75488b1b43 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:19.766626 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-5fb405cb-267c-4ea7-9afb-8b75488b1b43 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 15:47:19.766626 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:47:19.766626 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:47:19.766626 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:47:19.766626 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:47:19.766626 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:47:19.766626 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:47:19.766626 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:47:19.766626 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:47:19.766626 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:47:19.766626 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:47:19.766626 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:47:19.766626 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 15:47:19.766626 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 15:47:19.766626 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 15:47:19.766626 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 15:47:19.767641 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:47:19.767641 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:47:19.767641 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:47:19.767641 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:47:19.767641 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:47:19.767641 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:47:19.767641 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:47:19.767641 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Aug 30 15:47:19.767641 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:47:19.767641 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:47:19.767641 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:47:19.767641 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:47:19.767641 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:47:19.767641 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:47:19.767641 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:47:19.767641 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:47:19.767641 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 15:47:19.768524 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 15:47:19.768524 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 15:47:19.768524 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Aug 30 15:47:19.768707 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c12f547-88f5-4b5b-b66d-950089a52206 None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:19.775645 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 467/939] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 30 15:47:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 15:47:19.776748 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb405cb-267c-4ea7-9afb-8b75488b1b43 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:19.805293 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c817ce73-3a2e-4668-bc6c-dee8dbc84604 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:19.805846 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c817ce73-3a2e-4668-bc6c-dee8dbc84604 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:19.806115 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c817ce73-3a2e-4668-bc6c-dee8dbc84604 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:19.806381 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c817ce73-3a2e-4668-bc6c-dee8dbc84604 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:19.813959 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c817ce73-3a2e-4668-bc6c-dee8dbc84604 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:19.815714 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c817ce73-3a2e-4668-bc6c-dee8dbc84604 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:19.827907 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application [None req-c817ce73-3a2e-4668-bc6c-dee8dbc84604 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 15:47:19.827907 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:47:19.827907 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:47:19.827907 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:47:19.827907 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:47:19.827907 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:47:19.827907 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:47:19.827907 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:47:19.827907 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:47:19.827907 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:47:19.827907 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:47:19.827907 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:47:19.827907 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 15:47:19.827907 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 15:47:19.827907 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 15:47:19.828836 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 15:47:19.828836 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:47:19.828836 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:47:19.828836 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:47:19.828836 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:47:19.828836 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:47:19.828836 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:47:19.828836 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:47:19.828836 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application with Lock( Aug 30 15:47:19.828836 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:47:19.828836 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:47:19.828836 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:47:19.828836 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:47:19.828836 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:47:19.828836 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:47:19.828836 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:47:19.828836 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:47:19.829800 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 15:47:19.829800 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 15:47:19.829800 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 15:47:19.829800 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 15:47:19.829800 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 15:47:19.829800 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 15:47:19.829800 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 15:47:19.829800 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Aug 30 15:47:19.834107 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 473/940] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Aug 30 15:47:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 15:47:19.835162 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c817ce73-3a2e-4668-bc6c-dee8dbc84604 None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:19.874944 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:47:19.893638 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb405cb-267c-4ea7-9afb-8b75488b1b43 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:19.930797 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd405770-07db-4cc1-a160-860fd1008dc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:19.931422 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd405770-07db-4cc1-a160-860fd1008dc3 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:19.931702 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd405770-07db-4cc1-a160-860fd1008dc3 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:19.931958 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd405770-07db-4cc1-a160-860fd1008dc3 None admin] PATH_INFO: `/v3/domains` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:19.932600 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd405770-07db-4cc1-a160-860fd1008dc3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:47:19.932895 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd405770-07db-4cc1-a160-860fd1008dc3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:19.934292 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd405770-07db-4cc1-a160-860fd1008dc3 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:19.947055 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 468/941] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Aug 30 15:47:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:47:19.952875 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0e4b0e-927b-4954-b099-6a0e8d9e566c None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:19.953153 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0e4b0e-927b-4954-b099-6a0e8d9e566c None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:19.953441 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0e4b0e-927b-4954-b099-6a0e8d9e566c None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:19.954817 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 469/942] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 30 15:47:19 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 15:47:19.955366 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0e4b0e-927b-4954-b099-6a0e8d9e566c None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:19.962388 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c817ce73-3a2e-4668-bc6c-dee8dbc84604 None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:19.999389 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:47:20.000369 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782981d3-195c-4086-b569-d99950bd0c3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:20.000984 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782981d3-195c-4086-b569-d99950bd0c3d None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:20.001277 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782981d3-195c-4086-b569-d99950bd0c3d None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:20.001596 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782981d3-195c-4086-b569-d99950bd0c3d None admin] PATH_INFO: `/v3/projects` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:20.002212 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782981d3-195c-4086-b569-d99950bd0c3d None admin] RBAC: Adding query filter params (name=service) {{(pid=81512) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:47:20.002536 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782981d3-195c-4086-b569-d99950bd0c3d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:20.004219 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782981d3-195c-4086-b569-d99950bd0c3d None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:20.024534 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 474/943] 10.222.0.20 () {56 vars in 1112 bytes} [Tue Aug 30 15:47:19 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 15:47:20.025849 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e38dbdc-2d9b-4114-a8b5-0fa0e0548259 None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:20.026129 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e38dbdc-2d9b-4114-a8b5-0fa0e0548259 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:20.026393 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e38dbdc-2d9b-4114-a8b5-0fa0e0548259 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:20.052716 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-8e38dbdc-2d9b-4114-a8b5-0fa0e0548259 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:47:20.054394 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-8e38dbdc-2d9b-4114-a8b5-0fa0e0548259 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:47:20.083390 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0e4b0e-927b-4954-b099-6a0e8d9e566c None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:20.131529 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e254d8-ed23-49d3-98e0-6615df8600e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:20.132494 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e254d8-ed23-49d3-98e0-6615df8600e2 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:20.132965 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e254d8-ed23-49d3-98e0-6615df8600e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:20.133469 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e254d8-ed23-49d3-98e0-6615df8600e2 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:20.145175 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e254d8-ed23-49d3-98e0-6615df8600e2 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:20.147126 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e254d8-ed23-49d3-98e0-6615df8600e2 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:20.149669 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 475/944] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 30 15:47:20 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 15:47:20.160729 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-76e254d8-ed23-49d3-98e0-6615df8600e2 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 30 15:47:20.160729 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:47:20.160729 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:47:20.160729 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:47:20.160729 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:47:20.160729 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:47:20.160729 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:47:20.160729 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:47:20.160729 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:47:20.160729 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:47:20.160729 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:47:20.160729 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:47:20.160729 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 15:47:20.160729 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 15:47:20.160729 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 15:47:20.160729 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 15:47:20.161829 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:47:20.161829 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:47:20.161829 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 15:47:20.161829 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 15:47:20.161829 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 15:47:20.161829 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 15:47:20.161829 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:47:20.161829 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:47:20.161829 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:47:20.161829 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:47:20.161829 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:47:20.161829 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Aug 30 15:47:20.161829 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:47:20.161829 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:47:20.161829 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:47:20.161829 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:47:20.161829 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:47:20.161829 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:47:20.162794 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:47:20.162794 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:47:20.162794 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 15:47:20.162794 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 15:47:20.162794 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 15:47:20.162794 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 15:47:20.162794 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 15:47:20.162794 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 15:47:20.162794 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 30 15:47:20.162794 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Aug 30 15:47:20.165428 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 470/945] 10.222.0.20 () {56 vars in 1117 bytes} [Tue Aug 30 15:47:19 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 15:47:20.177263 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e38dbdc-2d9b-4114-a8b5-0fa0e0548259 None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:20.204282 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:47:20.281896 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e38dbdc-2d9b-4114-a8b5-0fa0e0548259 None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:20.318418 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5527de21-f88e-4950-8cb5-ff8ce53d2c27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:20.319388 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5527de21-f88e-4950-8cb5-ff8ce53d2c27 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:20.319809 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5527de21-f88e-4950-8cb5-ff8ce53d2c27 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:20.320209 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5527de21-f88e-4950-8cb5-ff8ce53d2c27 None admin] PATH_INFO: `/v3/users` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:20.321079 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5527de21-f88e-4950-8cb5-ff8ce53d2c27 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81512) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:47:20.321536 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5527de21-f88e-4950-8cb5-ff8ce53d2c27 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:20.323122 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5527de21-f88e-4950-8cb5-ff8ce53d2c27 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:20.341987 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 476/946] 10.222.0.20 () {56 vars in 1149 bytes} [Tue Aug 30 15:47:20 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:47:20.348021 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e254d8-ed23-49d3-98e0-6615df8600e2 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:20.429804 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5527de21-f88e-4950-8cb5-ff8ce53d2c27 None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:20.450590 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:47:20.459510 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e254d8-ed23-49d3-98e0-6615df8600e2 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:20.504999 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a055a2f9-6488-4731-abdf-d6551b24d45b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:20.505707 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a055a2f9-6488-4731-abdf-d6551b24d45b None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:20.505988 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a055a2f9-6488-4731-abdf-d6551b24d45b None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:20.506263 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a055a2f9-6488-4731-abdf-d6551b24d45b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:20.516075 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:47:20.522006 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a055a2f9-6488-4731-abdf-d6551b24d45b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:20.523483 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a055a2f9-6488-4731-abdf-d6551b24d45b None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:20.542241 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-a055a2f9-6488-4731-abdf-d6551b24d45b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 15:47:20.542241 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:47:20.542241 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:47:20.542241 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:47:20.542241 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:47:20.542241 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:47:20.542241 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:47:20.542241 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:47:20.542241 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:47:20.542241 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:47:20.542241 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:47:20.542241 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:47:20.542241 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 15:47:20.542241 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 15:47:20.542241 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 15:47:20.542241 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 15:47:20.543420 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:47:20.543420 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:47:20.543420 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:47:20.543420 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:47:20.543420 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:47:20.543420 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:47:20.543420 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:47:20.543420 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Aug 30 15:47:20.543420 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:47:20.543420 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:47:20.543420 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:47:20.543420 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:47:20.543420 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:47:20.543420 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:47:20.543420 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:47:20.543420 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:47:20.543420 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 15:47:20.544512 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 15:47:20.544512 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 15:47:20.544512 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Aug 30 15:47:20.546327 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5527de21-f88e-4950-8cb5-ff8ce53d2c27 None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:20.549861 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 471/947] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 30 15:47:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 15:47:20.551095 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a055a2f9-6488-4731-abdf-d6551b24d45b None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:20.594136 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fbb34d-7313-4d03-9ff1-24b4ed36f81f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:20.595168 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fbb34d-7313-4d03-9ff1-24b4ed36f81f None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:20.595458 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fbb34d-7313-4d03-9ff1-24b4ed36f81f None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:20.595736 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fbb34d-7313-4d03-9ff1-24b4ed36f81f None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:20.596381 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2fbb34d-7313-4d03-9ff1-24b4ed36f81f None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:20.598537 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2fbb34d-7313-4d03-9ff1-24b4ed36f81f None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:20.610936 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application [None req-b2fbb34d-7313-4d03-9ff1-24b4ed36f81f None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 30 15:47:20.610936 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:47:20.610936 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 15:47:20.610936 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 15:47:20.610936 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 15:47:20.610936 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 15:47:20.610936 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 15:47:20.610936 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 15:47:20.610936 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 30 15:47:20.610936 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Aug 30 15:47:20.610936 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 15:47:20.610936 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Aug 30 15:47:20.610936 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:47:20.610936 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:47:20.610936 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:47:20.610936 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:47:20.610936 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:47:20.612187 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:47:20.612187 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:47:20.612187 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:47:20.612187 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:47:20.612187 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:47:20.612187 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:47:20.612187 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 15:47:20.612187 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 15:47:20.612187 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 15:47:20.612187 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 15:47:20.612187 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:47:20.612187 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:47:20.612187 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:47:20.612187 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:47:20.612187 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:47:20.612187 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:47:20.613621 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:47:20.613621 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application with Lock( Aug 30 15:47:20.613621 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:47:20.613621 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:47:20.613621 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:47:20.613621 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:47:20.613621 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:47:20.613621 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:47:20.613621 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:47:20.613621 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:47:20.613621 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 15:47:20.613621 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 15:47:20.613621 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 30 15:47:20.613621 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Aug 30 15:47:20.613621 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 477/948] 10.222.0.20 () {56 vars in 1126 bytes} [Tue Aug 30 15:47:20 2022] GET /identity/v3/services/object-store => generated 93 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 15:47:20.618418 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fbb34d-7313-4d03-9ff1-24b4ed36f81f None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:20.665838 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a055a2f9-6488-4731-abdf-d6551b24d45b None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:20.700231 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581eccfa-00e9-4264-8f5f-495f5e797563 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:20.700846 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581eccfa-00e9-4264-8f5f-495f5e797563 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:20.701240 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581eccfa-00e9-4264-8f5f-495f5e797563 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:20.701519 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581eccfa-00e9-4264-8f5f-495f5e797563 None admin] PATH_INFO: `/v3/domains` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:20.702205 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581eccfa-00e9-4264-8f5f-495f5e797563 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:47:20.702508 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581eccfa-00e9-4264-8f5f-495f5e797563 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:20.704059 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581eccfa-00e9-4264-8f5f-495f5e797563 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:20.719311 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 472/949] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Aug 30 15:47:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:47:20.725064 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581eccfa-00e9-4264-8f5f-495f5e797563 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:20.736199 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:47:20.748553 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fbb34d-7313-4d03-9ff1-24b4ed36f81f None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:20.792386 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d58c646-d5d5-4930-8636-41dfb827a8d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:20.793108 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d58c646-d5d5-4930-8636-41dfb827a8d3 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:20.793505 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d58c646-d5d5-4930-8636-41dfb827a8d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:20.793844 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d58c646-d5d5-4930-8636-41dfb827a8d3 None admin] PATH_INFO: `/v3/services` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:20.794700 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d58c646-d5d5-4930-8636-41dfb827a8d3 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81512) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:47:20.795126 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d58c646-d5d5-4930-8636-41dfb827a8d3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:20.797197 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d58c646-d5d5-4930-8636-41dfb827a8d3 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:20.805297 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 478/950] 10.222.0.20 () {56 vars in 1122 bytes} [Tue Aug 30 15:47:20 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 15:47:20.809592 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d58c646-d5d5-4930-8636-41dfb827a8d3 None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:20.824351 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:47:20.855401 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581eccfa-00e9-4264-8f5f-495f5e797563 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:20.901071 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286ddb93-6eec-46d2-af17-67bdf67059d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:20.901859 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286ddb93-6eec-46d2-af17-67bdf67059d8 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:20.902208 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286ddb93-6eec-46d2-af17-67bdf67059d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:20.902691 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286ddb93-6eec-46d2-af17-67bdf67059d8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:20.911907 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286ddb93-6eec-46d2-af17-67bdf67059d8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:20.913765 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286ddb93-6eec-46d2-af17-67bdf67059d8 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:20.921352 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d58c646-d5d5-4930-8636-41dfb827a8d3 None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:20.928116 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-286ddb93-6eec-46d2-af17-67bdf67059d8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 15:47:20.928116 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:47:20.928116 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:47:20.928116 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:47:20.928116 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:47:20.928116 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:47:20.928116 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:47:20.928116 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:47:20.928116 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:47:20.928116 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:47:20.928116 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:47:20.928116 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:47:20.928116 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 15:47:20.928116 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 15:47:20.928116 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 15:47:20.929020 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 15:47:20.929020 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:47:20.929020 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:47:20.929020 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:47:20.929020 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:47:20.929020 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:47:20.929020 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:47:20.929020 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:47:20.929020 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Aug 30 15:47:20.929020 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:47:20.929020 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:47:20.929020 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:47:20.929020 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:47:20.929020 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:47:20.929020 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:47:20.929020 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:47:20.929020 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:47:20.930261 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 15:47:20.930261 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 15:47:20.930261 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 15:47:20.930261 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 15:47:20.930261 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 15:47:20.930261 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 15:47:20.930261 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 15:47:20.930261 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Aug 30 15:47:20.930261 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 473/951] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Aug 30 15:47:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 15:47:20.934814 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286ddb93-6eec-46d2-af17-67bdf67059d8 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:20.968877 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a414ecd4-1414-4e9b-8fbb-3b80a19d795d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:20.970148 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a414ecd4-1414-4e9b-8fbb-3b80a19d795d None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:20.971201 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a414ecd4-1414-4e9b-8fbb-3b80a19d795d None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:20.971896 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a414ecd4-1414-4e9b-8fbb-3b80a19d795d None admin] PATH_INFO: `/v3/services` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:20.972716 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a414ecd4-1414-4e9b-8fbb-3b80a19d795d None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81512) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:47:20.973440 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a414ecd4-1414-4e9b-8fbb-3b80a19d795d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:20.975693 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a414ecd4-1414-4e9b-8fbb-3b80a19d795d None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:20.987739 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 479/952] 10.222.0.20 () {56 vars in 1122 bytes} [Tue Aug 30 15:47:20 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:47:20.989012 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a414ecd4-1414-4e9b-8fbb-3b80a19d795d None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:21.025773 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:47:21.071478 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:47:21.075535 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286ddb93-6eec-46d2-af17-67bdf67059d8 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:21.108988 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a414ecd4-1414-4e9b-8fbb-3b80a19d795d None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:21.119649 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4f4549-b57a-42d5-87c7-ba85f5ca5f73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:21.120913 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4f4549-b57a-42d5-87c7-ba85f5ca5f73 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:21.121786 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4f4549-b57a-42d5-87c7-ba85f5ca5f73 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:21.122231 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4f4549-b57a-42d5-87c7-ba85f5ca5f73 None admin] PATH_INFO: `/v3/projects` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:21.123410 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4f4549-b57a-42d5-87c7-ba85f5ca5f73 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:47:21.123873 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4f4549-b57a-42d5-87c7-ba85f5ca5f73 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:21.126425 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4f4549-b57a-42d5-87c7-ba85f5ca5f73 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:21.144432 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 474/953] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Aug 30 15:47:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:47:21.146858 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9799b7-257c-4747-9cfa-2ab7610f7664 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:21.147698 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9799b7-257c-4747-9cfa-2ab7610f7664 None admin] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:21.148194 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9799b7-257c-4747-9cfa-2ab7610f7664 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:21.148665 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9799b7-257c-4747-9cfa-2ab7610f7664 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:21.150188 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4f4549-b57a-42d5-87c7-ba85f5ca5f73 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:21.150850 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9799b7-257c-4747-9cfa-2ab7610f7664 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:21.152241 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9799b7-257c-4747-9cfa-2ab7610f7664 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:21.189311 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 480/954] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Aug 30 15:47:20 2022] POST /identity/v3/endpoints => generated 349 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 15:47:21.270285 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4f4549-b57a-42d5-87c7-ba85f5ca5f73 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:21.307331 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:47:21.324357 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c368d87-e9a5-41cf-870c-f16c696aa3f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:21.325034 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c368d87-e9a5-41cf-870c-f16c696aa3f3 None admin] REQUEST_METHOD: `PUT` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:21.325378 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c368d87-e9a5-41cf-870c-f16c696aa3f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:21.325648 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c368d87-e9a5-41cf-870c-f16c696aa3f3 None admin] PATH_INFO: `/v3/projects/854fe71b49224dff9a4b1766a20ab266/users/07130d598131443486c2bc1c6673c45b/roles/a89e0730512d449ca32479022aaf58d1` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:21.327934 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c368d87-e9a5-41cf-870c-f16c696aa3f3 None admin] RBAC: Authorizing `identity:create_grant(project_id=854fe71b49224dff9a4b1766a20ab266, user_id=07130d598131443486c2bc1c6673c45b, role_id=a89e0730512d449ca32479022aaf58d1)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:21.329468 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c368d87-e9a5-41cf-870c-f16c696aa3f3 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:21.349084 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 475/955] 10.222.0.20 () {58 vars in 1439 bytes} [Tue Aug 30 15:47:21 2022] PUT /identity/v3/projects/854fe71b49224dff9a4b1766a20ab266/users/07130d598131443486c2bc1c6673c45b/roles/a89e0730512d449ca32479022aaf58d1 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 15:47:21.402268 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28277a03-2c5c-4651-861e-ae38043d2ecd None None] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:21.402580 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28277a03-2c5c-4651-861e-ae38043d2ecd None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:21.402816 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28277a03-2c5c-4651-861e-ae38043d2ecd None None] PATH_INFO: `/` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:21.404603 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 481/956] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 30 15:47:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 15:47:21.410799 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c012e6a-26e3-489e-8cc2-1807da75fe97 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:21.411074 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c012e6a-26e3-489e-8cc2-1807da75fe97 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:21.411335 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c012e6a-26e3-489e-8cc2-1807da75fe97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:21.432977 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-7c012e6a-26e3-489e-8cc2-1807da75fe97 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:47:21.434607 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-7c012e6a-26e3-489e-8cc2-1807da75fe97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:47:21.509489 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 476/957] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 30 15:47:21 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 15:47:21.772579 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28277a03-2c5c-4651-861e-ae38043d2ecd None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:21.880539 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28277a03-2c5c-4651-861e-ae38043d2ecd None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:21.914917 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:47:21.921306 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5d8656-560b-4778-9271-b4d7f0e82b3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:21.922141 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5d8656-560b-4778-9271-b4d7f0e82b3b None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:21.922482 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5d8656-560b-4778-9271-b4d7f0e82b3b None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:21.922904 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5d8656-560b-4778-9271-b4d7f0e82b3b None admin] PATH_INFO: `/v3/services/compute` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:21.923555 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5d8656-560b-4778-9271-b4d7f0e82b3b None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:21.925424 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5d8656-560b-4778-9271-b4d7f0e82b3b None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:21.934894 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application [None req-ba5d8656-560b-4778-9271-b4d7f0e82b3b None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 30 15:47:21.934894 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:47:21.934894 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 15:47:21.934894 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 15:47:21.934894 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 15:47:21.934894 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 15:47:21.934894 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 15:47:21.934894 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 15:47:21.934894 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 30 15:47:21.934894 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Aug 30 15:47:21.934894 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 15:47:21.934894 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Aug 30 15:47:21.934894 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:47:21.934894 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:47:21.934894 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:47:21.934894 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:47:21.934894 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:47:21.936214 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:47:21.936214 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:47:21.936214 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:47:21.936214 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:47:21.936214 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:47:21.936214 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:47:21.936214 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 15:47:21.936214 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 15:47:21.936214 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 15:47:21.936214 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 15:47:21.936214 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:47:21.936214 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:47:21.936214 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:47:21.936214 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:47:21.936214 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:47:21.936214 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:47:21.937137 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:47:21.937137 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application with Lock( Aug 30 15:47:21.937137 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:47:21.937137 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:47:21.937137 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:47:21.937137 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:47:21.937137 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:47:21.937137 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:47:21.937137 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:47:21.937137 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:47:21.937137 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 15:47:21.937137 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 15:47:21.937137 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 30 15:47:21.937137 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Aug 30 15:47:21.937137 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 482/958] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Aug 30 15:47:21 2022] GET /identity/v3/services/compute => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 15:47:21.945543 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c012e6a-26e3-489e-8cc2-1807da75fe97 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:22.050824 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c012e6a-26e3-489e-8cc2-1807da75fe97 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:22.064341 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:47:22.085701 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c00a72d-635a-4970-ac91-d7d4e61286be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:22.086655 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c00a72d-635a-4970-ac91-d7d4e61286be None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:22.087473 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c00a72d-635a-4970-ac91-d7d4e61286be None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:22.088016 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c00a72d-635a-4970-ac91-d7d4e61286be None admin] PATH_INFO: `/v3/services` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:22.088654 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c00a72d-635a-4970-ac91-d7d4e61286be None admin] RBAC: Adding query filter params (name=compute) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:47:22.088959 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c00a72d-635a-4970-ac91-d7d4e61286be None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:22.090377 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c00a72d-635a-4970-ac91-d7d4e61286be None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:22.095599 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 477/959] 10.222.0.20 () {56 vars in 1112 bytes} [Tue Aug 30 15:47:21 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:47:22.101337 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5d8656-560b-4778-9271-b4d7f0e82b3b None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:22.170922 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237570aa-f955-49da-874e-dc8ecb95211e None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:22.171549 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237570aa-f955-49da-874e-dc8ecb95211e None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:22.172009 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237570aa-f955-49da-874e-dc8ecb95211e None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:22.173830 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 478/960] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 30 15:47:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 15:47:22.183370 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06d7a0c-d30f-47a0-87f1-4fe468b2143a None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:22.184004 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06d7a0c-d30f-47a0-87f1-4fe468b2143a None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:22.184471 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06d7a0c-d30f-47a0-87f1-4fe468b2143a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:22.203578 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-b06d7a0c-d30f-47a0-87f1-4fe468b2143a None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:47:22.205132 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-b06d7a0c-d30f-47a0-87f1-4fe468b2143a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:47:22.206676 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5d8656-560b-4778-9271-b4d7f0e82b3b None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:22.247091 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e2e6ff-7bda-41fb-bd1b-4a774ce32a9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:22.248106 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e2e6ff-7bda-41fb-bd1b-4a774ce32a9f None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:22.248612 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e2e6ff-7bda-41fb-bd1b-4a774ce32a9f None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:22.249080 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e2e6ff-7bda-41fb-bd1b-4a774ce32a9f None admin] PATH_INFO: `/v3/services` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:22.250244 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e2e6ff-7bda-41fb-bd1b-4a774ce32a9f None admin] RBAC: Adding query filter params (type=compute) {{(pid=81512) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:47:22.251646 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e2e6ff-7bda-41fb-bd1b-4a774ce32a9f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:22.253578 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e2e6ff-7bda-41fb-bd1b-4a774ce32a9f None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:22.260298 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 483/961] 10.222.0.20 () {56 vars in 1112 bytes} [Tue Aug 30 15:47:22 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:47:22.296719 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 479/962] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 30 15:47:22 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 15:47:22.568342 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e2e6ff-7bda-41fb-bd1b-4a774ce32a9f None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:22.630302 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:47:22.673564 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e2e6ff-7bda-41fb-bd1b-4a774ce32a9f None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:22.708866 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669ee8bf-ddf6-4295-8de0-19b71d521d28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:22.709577 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669ee8bf-ddf6-4295-8de0-19b71d521d28 None admin] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:22.709848 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669ee8bf-ddf6-4295-8de0-19b71d521d28 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:22.710118 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669ee8bf-ddf6-4295-8de0-19b71d521d28 None admin] PATH_INFO: `/v3/users` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:22.710771 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669ee8bf-ddf6-4295-8de0-19b71d521d28 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:22.712200 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669ee8bf-ddf6-4295-8de0-19b71d521d28 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:22.727973 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.sql.core [None req-669ee8bf-ddf6-4295-8de0-19b71d521d28 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 30 15:47:22.727973 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 30 15:47:22.727973 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [parameters: {'user_id': '4f1ed7b21fcb4f7e85242351fe5e059c', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 30 15:47:22.727973 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81512) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 30 15:47:22.734803 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application [None req-669ee8bf-ddf6-4295-8de0-19b71d521d28 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 30 15:47:22.734803 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:47:22.734803 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 30 15:47:22.734803 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 30 15:47:22.734803 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 30 15:47:22.734803 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 30 15:47:22.734803 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 30 15:47:22.734803 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application result = self._query(query) Aug 30 15:47:22.734803 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 30 15:47:22.734803 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application conn.query(q) Aug 30 15:47:22.734803 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 30 15:47:22.734803 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 30 15:47:22.734803 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 30 15:47:22.734803 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application result.read() Aug 30 15:47:22.734803 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 30 15:47:22.735691 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 30 15:47:22.735691 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 30 15:47:22.735691 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 30 15:47:22.735691 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 30 15:47:22.735691 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 30 15:47:22.735691 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 30 15:47:22.735691 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 30 15:47:22.735691 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 30 15:47:22.735691 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Aug 30 15:47:22.735691 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 30 15:47:22.735691 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Aug 30 15:47:22.735691 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:47:22.735691 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 30 15:47:22.735691 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 30 15:47:22.735691 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Aug 30 15:47:22.735691 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Aug 30 15:47:22.735691 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 30 15:47:22.735691 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application next(self.gen) Aug 30 15:47:22.737096 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 30 15:47:22.737096 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application yield resource Aug 30 15:47:22.737096 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 30 15:47:22.737096 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application next(self.gen) Aug 30 15:47:22.737096 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 30 15:47:22.737096 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application self.session.rollback() Aug 30 15:47:22.737096 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 30 15:47:22.737096 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application self.force_reraise() Aug 30 15:47:22.737096 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 30 15:47:22.737096 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application raise self.value Aug 30 15:47:22.737096 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 30 15:47:22.737096 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 30 15:47:22.737096 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 30 15:47:22.737096 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application session.commit() Aug 30 15:47:22.737096 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 30 15:47:22.737096 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 30 15:47:22.737096 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 30 15:47:22.739219 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application self._prepare_impl() Aug 30 15:47:22.739219 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 30 15:47:22.739219 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application self.session.flush() Aug 30 15:47:22.739219 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 30 15:47:22.739219 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application self._flush(objects) Aug 30 15:47:22.739219 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 30 15:47:22.739219 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 30 15:47:22.739219 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 30 15:47:22.739219 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application compat.raise_( Aug 30 15:47:22.739219 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 30 15:47:22.739219 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application raise exception Aug 30 15:47:22.739219 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 30 15:47:22.739219 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application flush_context.execute() Aug 30 15:47:22.739219 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 30 15:47:22.739219 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application rec.execute(self) Aug 30 15:47:22.739219 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 30 15:47:22.739219 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 30 15:47:22.745401 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 30 15:47:22.745401 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 30 15:47:22.745401 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Aug 30 15:47:22.745401 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application result = connection._execute_20( Aug 30 15:47:22.745401 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 30 15:47:22.745401 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 30 15:47:22.745401 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 30 15:47:22.745401 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 30 15:47:22.745401 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 30 15:47:22.745401 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 30 15:47:22.745401 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 30 15:47:22.745401 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 30 15:47:22.745401 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 30 15:47:22.745401 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 30 15:47:22.745401 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 30 15:47:22.745401 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application raise exception Aug 30 15:47:22.746283 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 30 15:47:22.746283 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 30 15:47:22.746283 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 30 15:47:22.746283 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 30 15:47:22.746283 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 30 15:47:22.746283 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application result = self._query(query) Aug 30 15:47:22.746283 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 30 15:47:22.746283 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application conn.query(q) Aug 30 15:47:22.746283 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 30 15:47:22.746283 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 30 15:47:22.746283 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 30 15:47:22.746283 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application result.read() Aug 30 15:47:22.746283 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 30 15:47:22.746283 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 30 15:47:22.746283 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 30 15:47:22.746283 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 30 15:47:22.747143 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 30 15:47:22.747143 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 30 15:47:22.747143 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 30 15:47:22.747143 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 30 15:47:22.747143 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 30 15:47:22.747143 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 30 15:47:22.747143 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application [parameters: {'user_id': '4f1ed7b21fcb4f7e85242351fe5e059c', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 30 15:47:22.747143 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 30 15:47:22.747143 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Aug 30 15:47:22.747143 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 15:47:22.747143 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Aug 30 15:47:22.747143 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:47:22.747143 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:47:22.747143 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:47:22.747143 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:47:22.748072 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:47:22.748072 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:47:22.748072 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:47:22.748072 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:47:22.748072 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:47:22.748072 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:47:22.748072 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:47:22.748072 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Aug 30 15:47:22.748072 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Aug 30 15:47:22.748072 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:47:22.748072 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:47:22.748072 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 15:47:22.748072 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 15:47:22.748072 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 15:47:22.748072 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 15:47:22.748072 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Aug 30 15:47:22.748072 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Aug 30 15:47:22.749848 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Aug 30 15:47:22.749848 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Aug 30 15:47:22.749848 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 30 15:47:22.749848 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 30 15:47:22.749848 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 30 15:47:22.749848 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Aug 30 15:47:22.749848 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 484/963] 10.222.0.20 () {60 vars in 1131 bytes} [Tue Aug 30 15:47:22 2022] POST /identity/v3/users => generated 160 bytes in 168 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 30 15:47:22.751592 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06d7a0c-d30f-47a0-87f1-4fe468b2143a None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:22.858674 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06d7a0c-d30f-47a0-87f1-4fe468b2143a None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:22.866591 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:47:22.894949 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd1191e-c7a5-45af-a259-6f79c36c8da3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:22.895596 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd1191e-c7a5-45af-a259-6f79c36c8da3 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:22.895870 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd1191e-c7a5-45af-a259-6f79c36c8da3 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:22.896138 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd1191e-c7a5-45af-a259-6f79c36c8da3 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:22.903730 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd1191e-c7a5-45af-a259-6f79c36c8da3 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:22.905185 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd1191e-c7a5-45af-a259-6f79c36c8da3 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:22.918388 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-4cd1191e-c7a5-45af-a259-6f79c36c8da3 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 30 15:47:22.918388 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:47:22.918388 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:47:22.918388 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:47:22.918388 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:47:22.918388 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:47:22.918388 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:47:22.918388 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:47:22.918388 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:47:22.918388 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:47:22.918388 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:47:22.918388 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:47:22.918388 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 15:47:22.918388 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 15:47:22.918388 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 15:47:22.918388 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 15:47:22.919359 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:47:22.919359 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:47:22.919359 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 15:47:22.919359 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 15:47:22.919359 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 15:47:22.919359 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 15:47:22.919359 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:47:22.919359 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:47:22.919359 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:47:22.919359 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:47:22.919359 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:47:22.919359 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Aug 30 15:47:22.919359 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:47:22.919359 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:47:22.919359 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:47:22.919359 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:47:22.919359 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:47:22.919359 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:47:22.920424 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:47:22.920424 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:47:22.920424 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 15:47:22.920424 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 15:47:22.920424 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 15:47:22.920424 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 15:47:22.920424 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 15:47:22.920424 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 15:47:22.920424 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 30 15:47:22.920424 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Aug 30 15:47:22.920424 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 480/964] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Aug 30 15:47:22 2022] GET /identity/v3/users/cinder => generated 84 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 15:47:22.927764 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669ee8bf-ddf6-4295-8de0-19b71d521d28 None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:23.035398 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669ee8bf-ddf6-4295-8de0-19b71d521d28 None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:23.055679 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:47:23.070384 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a681714-45e0-47cb-9edc-3fcb84f35d7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:23.070991 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a681714-45e0-47cb-9edc-3fcb84f35d7e None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:23.071264 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a681714-45e0-47cb-9edc-3fcb84f35d7e None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:23.071529 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a681714-45e0-47cb-9edc-3fcb84f35d7e None admin] PATH_INFO: `/v3/users` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:23.072307 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a681714-45e0-47cb-9edc-3fcb84f35d7e None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81512) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:47:23.072601 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a681714-45e0-47cb-9edc-3fcb84f35d7e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:23.074263 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a681714-45e0-47cb-9edc-3fcb84f35d7e None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:23.113324 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 485/965] 10.222.0.20 () {56 vars in 1101 bytes} [Tue Aug 30 15:47:22 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:47:23.269770 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ae9885-91a7-4180-85b1-99cae7a38f16 None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:23.270174 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ae9885-91a7-4180-85b1-99cae7a38f16 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:23.270348 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ae9885-91a7-4180-85b1-99cae7a38f16 None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:23.271797 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 481/966] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 30 15:47:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 15:47:23.277189 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff3f6e6-a1bc-4690-9f00-a93980ba15b9 None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:23.277536 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff3f6e6-a1bc-4690-9f00-a93980ba15b9 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:23.277819 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff3f6e6-a1bc-4690-9f00-a93980ba15b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:23.297620 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-cff3f6e6-a1bc-4690-9f00-a93980ba15b9 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:47:23.299456 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-cff3f6e6-a1bc-4690-9f00-a93980ba15b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:47:23.320372 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d708d775-6b47-452c-8f4f-bbc4b73be7be None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:23.320682 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d708d775-6b47-452c-8f4f-bbc4b73be7be None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:23.321466 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d708d775-6b47-452c-8f4f-bbc4b73be7be None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:23.322923 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 482/967] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 30 15:47:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 15:47:23.329588 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ef80d7-6a69-4432-9372-11e4121837a8 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:23.329866 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ef80d7-6a69-4432-9372-11e4121837a8 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:23.330133 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ef80d7-6a69-4432-9372-11e4121837a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:23.355929 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-57ef80d7-6a69-4432-9372-11e4121837a8 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:47:23.357411 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-57ef80d7-6a69-4432-9372-11e4121837a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:47:23.372206 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 486/968] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 30 15:47:23 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 15:47:23.433262 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 483/969] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 30 15:47:23 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 15:47:23.637454 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff3f6e6-a1bc-4690-9f00-a93980ba15b9 None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:23.695309 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ef80d7-6a69-4432-9372-11e4121837a8 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:23.752888 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff3f6e6-a1bc-4690-9f00-a93980ba15b9 None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:23.763615 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:47:23.792950 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89294c24-fc28-4a24-a264-6db61c88c874 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:23.793878 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89294c24-fc28-4a24-a264-6db61c88c874 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:23.794157 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89294c24-fc28-4a24-a264-6db61c88c874 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:23.794425 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89294c24-fc28-4a24-a264-6db61c88c874 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:23.795218 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89294c24-fc28-4a24-a264-6db61c88c874 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:23.795685 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:47:23.797253 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89294c24-fc28-4a24-a264-6db61c88c874 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:23.801303 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ef80d7-6a69-4432-9372-11e4121837a8 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:23.808097 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application [None req-89294c24-fc28-4a24-a264-6db61c88c874 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 30 15:47:23.808097 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:47:23.808097 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 15:47:23.808097 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 15:47:23.808097 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 15:47:23.808097 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 15:47:23.808097 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 15:47:23.808097 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 15:47:23.808097 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 30 15:47:23.808097 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Aug 30 15:47:23.808097 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 15:47:23.808097 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Aug 30 15:47:23.808097 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:47:23.808097 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:47:23.808097 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:47:23.808097 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:47:23.808097 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:47:23.809082 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:47:23.809082 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:47:23.809082 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:47:23.809082 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:47:23.809082 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:47:23.809082 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:47:23.809082 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 15:47:23.809082 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 15:47:23.809082 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 15:47:23.809082 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 15:47:23.809082 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:47:23.809082 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:47:23.809082 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:47:23.809082 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:47:23.809082 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:47:23.809082 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:47:23.809957 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:47:23.809957 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application with Lock( Aug 30 15:47:23.809957 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:47:23.809957 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:47:23.809957 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:47:23.809957 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:47:23.809957 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:47:23.809957 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:47:23.809957 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:47:23.809957 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:47:23.809957 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 15:47:23.809957 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 15:47:23.809957 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 30 15:47:23.809957 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Aug 30 15:47:23.811409 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 487/970] 10.222.0.20 () {56 vars in 1126 bytes} [Tue Aug 30 15:47:23 2022] GET /identity/v3/services/object-store => generated 93 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 15:47:23.815449 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89294c24-fc28-4a24-a264-6db61c88c874 None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:23.838950 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ce0e03-bc6b-458d-af91-35cdcb4600f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:23.839984 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ce0e03-bc6b-458d-af91-35cdcb4600f7 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:23.840276 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ce0e03-bc6b-458d-af91-35cdcb4600f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:23.840564 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ce0e03-bc6b-458d-af91-35cdcb4600f7 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:23.851607 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ce0e03-bc6b-458d-af91-35cdcb4600f7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 15:47:23.856483 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-c2ce0e03-bc6b-458d-af91-35cdcb4600f7 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 30 15:47:23.856483 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:47:23.856483 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:47:23.856483 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:47:23.856483 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:47:23.856483 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:47:23.856483 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:47:23.856483 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:47:23.856483 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:47:23.856483 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:47:23.856483 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:47:23.856483 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:47:23.856483 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 15:47:23.856483 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 15:47:23.856483 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 15:47:23.856483 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 15:47:23.857693 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 15:47:23.857693 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 15:47:23.857693 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 15:47:23.857693 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 15:47:23.857693 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:47:23.857693 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:47:23.857693 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:47:23.857693 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:47:23.857693 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:47:23.857693 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:47:23.857693 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:47:23.857693 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Aug 30 15:47:23.857693 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:47:23.857693 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:47:23.857693 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:47:23.857693 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:47:23.858580 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:47:23.858580 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:47:23.858580 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:47:23.858580 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:47:23.858580 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 15:47:23.858580 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 15:47:23.858580 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 15:47:23.858580 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 15:47:23.858580 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 15:47:23.858580 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 15:47:23.858580 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 30 15:47:23.858580 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Aug 30 15:47:23.858580 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 484/971] 10.222.0.20 () {56 vars in 1120 bytes} [Tue Aug 30 15:47:23 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 15:47:23.864330 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ce0e03-bc6b-458d-af91-35cdcb4600f7 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:23.923856 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89294c24-fc28-4a24-a264-6db61c88c874 None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:23.958577 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86065ab-d73d-4c6b-a7a5-e1b3f83301b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:23.959259 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86065ab-d73d-4c6b-a7a5-e1b3f83301b6 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:23.959433 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86065ab-d73d-4c6b-a7a5-e1b3f83301b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:23.959700 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86065ab-d73d-4c6b-a7a5-e1b3f83301b6 None admin] PATH_INFO: `/v3/services` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:23.960320 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86065ab-d73d-4c6b-a7a5-e1b3f83301b6 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81512) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:47:23.960642 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86065ab-d73d-4c6b-a7a5-e1b3f83301b6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:23.962076 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86065ab-d73d-4c6b-a7a5-e1b3f83301b6 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:23.967362 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 488/972] 10.222.0.20 () {56 vars in 1122 bytes} [Tue Aug 30 15:47:23 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 15:47:23.972983 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86065ab-d73d-4c6b-a7a5-e1b3f83301b6 None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:23.974738 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ce0e03-bc6b-458d-af91-35cdcb4600f7 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:24.012958 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418f242a-7c52-4466-9d6f-1f4ec7f7c7b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:24.013634 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418f242a-7c52-4466-9d6f-1f4ec7f7c7b0 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:24.013903 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418f242a-7c52-4466-9d6f-1f4ec7f7c7b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:24.014161 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418f242a-7c52-4466-9d6f-1f4ec7f7c7b0 None admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:24.014827 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418f242a-7c52-4466-9d6f-1f4ec7f7c7b0 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:47:24.015122 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418f242a-7c52-4466-9d6f-1f4ec7f7c7b0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:24.016441 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:47:24.016567 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418f242a-7c52-4466-9d6f-1f4ec7f7c7b0 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:24.025648 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 485/973] 10.222.0.20 () {56 vars in 1115 bytes} [Tue Aug 30 15:47:23 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:47:24.031090 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418f242a-7c52-4466-9d6f-1f4ec7f7c7b0 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:24.053033 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:47:24.093362 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86065ab-d73d-4c6b-a7a5-e1b3f83301b6 None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:24.130281 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd6302c-9a65-463b-9cc3-01ddda27300d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:24.130886 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd6302c-9a65-463b-9cc3-01ddda27300d None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:24.131160 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd6302c-9a65-463b-9cc3-01ddda27300d None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:24.131425 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd6302c-9a65-463b-9cc3-01ddda27300d None admin] PATH_INFO: `/v3/services` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:24.132194 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd6302c-9a65-463b-9cc3-01ddda27300d None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81512) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:47:24.132529 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd6302c-9a65-463b-9cc3-01ddda27300d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:24.133939 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd6302c-9a65-463b-9cc3-01ddda27300d None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:24.143817 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 489/974] 10.222.0.20 () {56 vars in 1122 bytes} [Tue Aug 30 15:47:23 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 15:47:24.144640 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd6302c-9a65-463b-9cc3-01ddda27300d None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:24.145948 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418f242a-7c52-4466-9d6f-1f4ec7f7c7b0 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:24.179735 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96817ad3-856a-44c5-86c8-29c039c3983d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:24.180296 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96817ad3-856a-44c5-86c8-29c039c3983d None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:24.180574 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96817ad3-856a-44c5-86c8-29c039c3983d None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:24.180844 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96817ad3-856a-44c5-86c8-29c039c3983d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:24.191515 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96817ad3-856a-44c5-86c8-29c039c3983d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:24.192999 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96817ad3-856a-44c5-86c8-29c039c3983d None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:24.206954 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-96817ad3-856a-44c5-86c8-29c039c3983d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 15:47:24.206954 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:47:24.206954 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:47:24.206954 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:47:24.206954 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:47:24.206954 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:47:24.206954 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:47:24.206954 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:47:24.206954 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:47:24.206954 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:47:24.206954 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:47:24.206954 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:47:24.206954 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 15:47:24.206954 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 15:47:24.206954 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 15:47:24.206954 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 15:47:24.207954 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:47:24.207954 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:47:24.207954 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:47:24.207954 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:47:24.207954 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:47:24.207954 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:47:24.207954 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:47:24.207954 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Aug 30 15:47:24.207954 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:47:24.207954 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:47:24.207954 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:47:24.207954 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:47:24.207954 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:47:24.207954 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:47:24.207954 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:47:24.207954 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:47:24.207954 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 15:47:24.208877 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 15:47:24.208877 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 15:47:24.208877 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Aug 30 15:47:24.208877 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 486/975] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 30 15:47:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 15:47:24.209665 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cc5383-d587-47f3-9aca-7cb83b1d8ce2 None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:24.209665 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cc5383-d587-47f3-9aca-7cb83b1d8ce2 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:24.209916 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cc5383-d587-47f3-9aca-7cb83b1d8ce2 None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:24.213366 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 487/976] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 30 15:47:24 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 15:47:24.214121 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cc5383-d587-47f3-9aca-7cb83b1d8ce2 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:24.241823 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:47:24.244763 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd6302c-9a65-463b-9cc3-01ddda27300d None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:24.280184 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b5da95-402a-4e3a-8efe-cf0f9776481f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:24.280799 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b5da95-402a-4e3a-8efe-cf0f9776481f None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:24.281077 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b5da95-402a-4e3a-8efe-cf0f9776481f None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:24.281383 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b5da95-402a-4e3a-8efe-cf0f9776481f None admin] PATH_INFO: `/v3/endpoints` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:24.282089 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b5da95-402a-4e3a-8efe-cf0f9776481f None admin] RBAC: Adding query filter params (interface=admin, service_id=cbb1b96d587744728e67a883846db270, region_id=RegionOne) {{(pid=81512) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:47:24.282399 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b5da95-402a-4e3a-8efe-cf0f9776481f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:24.283827 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b5da95-402a-4e3a-8efe-cf0f9776481f None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:24.288994 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 490/977] 10.222.0.20 () {56 vars in 1249 bytes} [Tue Aug 30 15:47:24 2022] GET /identity/v3/endpoints?service_id=cbb1b96d587744728e67a883846db270&interface=admin®ion_id=RegionOne => generated 193 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 15:47:24.289984 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cc9f47-31d2-4cf9-a2d6-3d6c26bbc684 None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:24.290264 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cc9f47-31d2-4cf9-a2d6-3d6c26bbc684 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:24.290532 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cc9f47-31d2-4cf9-a2d6-3d6c26bbc684 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:24.291310 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:47:24.309356 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-d5cc9f47-31d2-4cf9-a2d6-3d6c26bbc684 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:47:24.310555 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-d5cc9f47-31d2-4cf9-a2d6-3d6c26bbc684 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:47:24.324628 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cc5383-d587-47f3-9aca-7cb83b1d8ce2 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:24.356861 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61ef671-fbb1-474f-ae59-7f437e490575 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:24.357531 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61ef671-fbb1-474f-ae59-7f437e490575 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:24.357804 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61ef671-fbb1-474f-ae59-7f437e490575 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:24.358067 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61ef671-fbb1-474f-ae59-7f437e490575 None admin] PATH_INFO: `/v3/domains` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:24.358702 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61ef671-fbb1-474f-ae59-7f437e490575 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:47:24.358995 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61ef671-fbb1-474f-ae59-7f437e490575 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:24.360357 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61ef671-fbb1-474f-ae59-7f437e490575 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:24.372128 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 488/978] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Aug 30 15:47:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:47:24.372943 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61ef671-fbb1-474f-ae59-7f437e490575 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:24.393259 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 491/979] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 30 15:47:24 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 15:47:24.393259 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cc9f47-31d2-4cf9-a2d6-3d6c26bbc684 None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:24.478558 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:47:24.481100 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61ef671-fbb1-474f-ae59-7f437e490575 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:24.486653 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cc9f47-31d2-4cf9-a2d6-3d6c26bbc684 None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:24.512648 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:47:24.520227 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8924a987-7335-4e71-8d3f-c60ee6cb74d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:24.520887 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8924a987-7335-4e71-8d3f-c60ee6cb74d4 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:24.521161 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8924a987-7335-4e71-8d3f-c60ee6cb74d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:24.521479 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8924a987-7335-4e71-8d3f-c60ee6cb74d4 None admin] PATH_INFO: `/v3/services` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:24.522083 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8924a987-7335-4e71-8d3f-c60ee6cb74d4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:24.522942 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e122f7-28bb-4a91-b75c-cf337fd14160 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:24.523462 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8924a987-7335-4e71-8d3f-c60ee6cb74d4 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:24.523980 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e122f7-28bb-4a91-b75c-cf337fd14160 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:24.524426 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e122f7-28bb-4a91-b75c-cf337fd14160 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:24.524839 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e122f7-28bb-4a91-b75c-cf337fd14160 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:24.529448 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 489/980] 10.222.0.20 () {56 vars in 1087 bytes} [Tue Aug 30 15:47:24 2022] GET /identity/v3/services => generated 1497 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 15:47:24.532400 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e122f7-28bb-4a91-b75c-cf337fd14160 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:24.533903 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e122f7-28bb-4a91-b75c-cf337fd14160 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:24.547050 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application [None req-68e122f7-28bb-4a91-b75c-cf337fd14160 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 30 15:47:24.547050 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:47:24.547050 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:47:24.547050 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:47:24.547050 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:47:24.547050 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:47:24.547050 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:47:24.547050 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:47:24.547050 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:47:24.547050 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:47:24.547050 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:47:24.547050 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:47:24.547050 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 15:47:24.547050 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 15:47:24.547050 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 15:47:24.547050 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 15:47:24.548093 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:47:24.548093 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:47:24.548093 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 15:47:24.548093 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 15:47:24.548093 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 15:47:24.548093 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 15:47:24.548093 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:47:24.548093 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:47:24.548093 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:47:24.548093 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:47:24.548093 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:47:24.548093 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application with Lock( Aug 30 15:47:24.548093 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:47:24.548093 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:47:24.548093 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:47:24.548093 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:47:24.548093 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:47:24.548093 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:47:24.549089 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:47:24.549089 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:47:24.549089 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 15:47:24.549089 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 15:47:24.549089 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 15:47:24.549089 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 15:47:24.549089 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 15:47:24.549089 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 15:47:24.549089 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 30 15:47:24.549089 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Aug 30 15:47:24.549089 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 492/981] 10.222.0.20 () {56 vars in 1117 bytes} [Tue Aug 30 15:47:24 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 15:47:24.554889 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8924a987-7335-4e71-8d3f-c60ee6cb74d4 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:24.636402 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e122f7-28bb-4a91-b75c-cf337fd14160 None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:24.680347 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8924a987-7335-4e71-8d3f-c60ee6cb74d4 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:24.719508 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7dec7b9-7750-4f7e-aac9-b37533a5b04c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:24.720213 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dec7b9-7750-4f7e-aac9-b37533a5b04c None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:24.720393 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dec7b9-7750-4f7e-aac9-b37533a5b04c None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:24.720662 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dec7b9-7750-4f7e-aac9-b37533a5b04c None admin] PATH_INFO: `/v3/users` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:24.721488 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7dec7b9-7750-4f7e-aac9-b37533a5b04c None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:47:24.721796 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7dec7b9-7750-4f7e-aac9-b37533a5b04c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:24.723263 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7dec7b9-7750-4f7e-aac9-b37533a5b04c None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:24.742345 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 490/982] 10.222.0.20 () {56 vars in 1149 bytes} [Tue Aug 30 15:47:24 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 15:47:24.748158 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7dec7b9-7750-4f7e-aac9-b37533a5b04c None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:24.762852 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e122f7-28bb-4a91-b75c-cf337fd14160 None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:24.798725 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29d03c0-b250-4f78-96ad-85db0908a912 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:24.799328 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29d03c0-b250-4f78-96ad-85db0908a912 None admin] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:24.799600 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29d03c0-b250-4f78-96ad-85db0908a912 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:24.799866 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29d03c0-b250-4f78-96ad-85db0908a912 None admin] PATH_INFO: `/v3/services` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:24.800516 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29d03c0-b250-4f78-96ad-85db0908a912 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:24.801913 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29d03c0-b250-4f78-96ad-85db0908a912 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:24.804398 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:47:24.818697 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 493/983] 10.222.0.20 () {60 vars in 1141 bytes} [Tue Aug 30 15:47:24 2022] POST /identity/v3/services => generated 242 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 15:47:24.859816 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7dec7b9-7750-4f7e-aac9-b37533a5b04c None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:24.907056 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bc4383-1d79-453b-9cf0-e59d7a5e1c69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:24.907725 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bc4383-1d79-453b-9cf0-e59d7a5e1c69 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:24.907997 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bc4383-1d79-453b-9cf0-e59d7a5e1c69 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:24.908257 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bc4383-1d79-453b-9cf0-e59d7a5e1c69 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:24.986419 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13bc4383-1d79-453b-9cf0-e59d7a5e1c69 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:24.987877 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13bc4383-1d79-453b-9cf0-e59d7a5e1c69 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:25.038634 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:47:25.092028 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 491/984] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Aug 30 15:47:24 2022] GET /identity/v3/auth/tokens => generated 1570 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 30 15:47:25.097534 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29d03c0-b250-4f78-96ad-85db0908a912 None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:25.114682 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:47:25.205371 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29d03c0-b250-4f78-96ad-85db0908a912 None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:25.238678 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d752154b-5e8d-4513-9beb-200112ab91c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:25.239309 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d752154b-5e8d-4513-9beb-200112ab91c3 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:25.239588 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d752154b-5e8d-4513-9beb-200112ab91c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:25.239857 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d752154b-5e8d-4513-9beb-200112ab91c3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:25.250916 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d752154b-5e8d-4513-9beb-200112ab91c3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:25.252379 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d752154b-5e8d-4513-9beb-200112ab91c3 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:25.266581 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application [None req-d752154b-5e8d-4513-9beb-200112ab91c3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 15:47:25.266581 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:47:25.266581 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:47:25.266581 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:47:25.266581 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:47:25.266581 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:47:25.266581 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:47:25.266581 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:47:25.266581 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:47:25.266581 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:47:25.266581 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:47:25.266581 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:47:25.266581 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 15:47:25.266581 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 15:47:25.266581 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 15:47:25.266581 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 15:47:25.267567 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:47:25.267567 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:47:25.267567 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:47:25.267567 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:47:25.267567 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:47:25.267567 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:47:25.267567 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:47:25.267567 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application with Lock( Aug 30 15:47:25.267567 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:47:25.267567 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:47:25.267567 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:47:25.267567 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:47:25.267567 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:47:25.267567 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:47:25.267567 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:47:25.267567 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:47:25.267567 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 15:47:25.268499 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 15:47:25.268499 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 15:47:25.268499 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Aug 30 15:47:25.269366 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 494/985] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 30 15:47:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 15:47:25.273481 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bc4383-1d79-453b-9cf0-e59d7a5e1c69 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:25.379350 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bc4383-1d79-453b-9cf0-e59d7a5e1c69 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:25.413436 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448cf297-95f1-448b-bbe2-8cf20185f021 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:25.414033 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448cf297-95f1-448b-bbe2-8cf20185f021 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:25.414302 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448cf297-95f1-448b-bbe2-8cf20185f021 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:25.414564 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448cf297-95f1-448b-bbe2-8cf20185f021 None admin] PATH_INFO: `/v3/domains` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:25.415198 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448cf297-95f1-448b-bbe2-8cf20185f021 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:47:25.415488 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448cf297-95f1-448b-bbe2-8cf20185f021 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:25.416837 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448cf297-95f1-448b-bbe2-8cf20185f021 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:25.428125 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 492/986] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Aug 30 15:47:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:47:25.433719 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d752154b-5e8d-4513-9beb-200112ab91c3 None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:25.504932 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:47:25.542352 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d752154b-5e8d-4513-9beb-200112ab91c3 None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:25.577597 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71426bc-5dd5-459f-abc7-fd2f65d37ad3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:25.578224 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71426bc-5dd5-459f-abc7-fd2f65d37ad3 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:25.578504 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71426bc-5dd5-459f-abc7-fd2f65d37ad3 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:25.578771 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71426bc-5dd5-459f-abc7-fd2f65d37ad3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:25.585566 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71426bc-5dd5-459f-abc7-fd2f65d37ad3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:25.587016 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71426bc-5dd5-459f-abc7-fd2f65d37ad3 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:25.599063 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application [None req-c71426bc-5dd5-459f-abc7-fd2f65d37ad3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 15:47:25.599063 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:47:25.599063 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:47:25.599063 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:47:25.599063 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:47:25.599063 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:47:25.599063 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:47:25.599063 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:47:25.599063 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:47:25.599063 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:47:25.599063 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:47:25.599063 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:47:25.599063 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 15:47:25.599063 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 15:47:25.599063 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 15:47:25.599974 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 15:47:25.599974 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:47:25.599974 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:47:25.599974 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:47:25.599974 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:47:25.599974 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:47:25.599974 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:47:25.599974 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:47:25.599974 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application with Lock( Aug 30 15:47:25.599974 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:47:25.599974 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:47:25.599974 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:47:25.599974 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:47:25.599974 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:47:25.599974 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:47:25.599974 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:47:25.599974 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:47:25.600881 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 15:47:25.600881 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 15:47:25.600881 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 15:47:25.600881 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 15:47:25.600881 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 15:47:25.600881 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 15:47:25.600881 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 15:47:25.600881 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Aug 30 15:47:25.600881 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 495/987] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Aug 30 15:47:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 15:47:25.606206 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448cf297-95f1-448b-bbe2-8cf20185f021 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:25.618632 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:47:25.715887 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448cf297-95f1-448b-bbe2-8cf20185f021 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:25.751059 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca33d98-ef9f-4866-8324-e86180a1d61a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:25.751653 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca33d98-ef9f-4866-8324-e86180a1d61a None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:25.751923 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca33d98-ef9f-4866-8324-e86180a1d61a None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:25.752184 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca33d98-ef9f-4866-8324-e86180a1d61a None admin] PATH_INFO: `/v3/projects` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:25.752864 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca33d98-ef9f-4866-8324-e86180a1d61a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:47:25.753207 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca33d98-ef9f-4866-8324-e86180a1d61a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:25.754605 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca33d98-ef9f-4866-8324-e86180a1d61a None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:25.771346 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 493/988] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Aug 30 15:47:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:47:25.772765 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71426bc-5dd5-459f-abc7-fd2f65d37ad3 None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:25.877039 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71426bc-5dd5-459f-abc7-fd2f65d37ad3 None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:25.902081 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:47:25.911944 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f01125-4256-465d-9204-3d8dc9f6c80a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:25.912566 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f01125-4256-465d-9204-3d8dc9f6c80a None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:25.912846 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f01125-4256-465d-9204-3d8dc9f6c80a None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:25.913120 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f01125-4256-465d-9204-3d8dc9f6c80a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:25.913907 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f01125-4256-465d-9204-3d8dc9f6c80a None admin] RBAC: Adding query filter params (role.id=a89e0730512d449ca32479022aaf58d1, scope.project.id=854fe71b49224dff9a4b1766a20ab266, user.id=07130d598131443486c2bc1c6673c45b) {{(pid=81512) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:47:25.914216 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f01125-4256-465d-9204-3d8dc9f6c80a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:25.915698 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f01125-4256-465d-9204-3d8dc9f6c80a None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:25.922632 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 496/989] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Aug 30 15:47:25 2022] GET /identity/v3/role_assignments?user.id=07130d598131443486c2bc1c6673c45b&scope.project.id=854fe71b49224dff9a4b1766a20ab266&role.id=a89e0730512d449ca32479022aaf58d1 => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:47:26.458552 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5031355-462a-4c35-a1cb-f16c9511ee56 None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:26.459107 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5031355-462a-4c35-a1cb-f16c9511ee56 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:26.459107 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5031355-462a-4c35-a1cb-f16c9511ee56 None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:26.460610 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 494/990] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 30 15:47:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 15:47:26.465761 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8bd7a0-fffe-4d6b-979b-368965d04d18 None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:26.466182 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8bd7a0-fffe-4d6b-979b-368965d04d18 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:26.466326 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8bd7a0-fffe-4d6b-979b-368965d04d18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:26.485766 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-bf8bd7a0-fffe-4d6b-979b-368965d04d18 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:47:26.486956 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-bf8bd7a0-fffe-4d6b-979b-368965d04d18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:47:26.556023 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 497/991] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 30 15:47:26 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 15:47:26.745174 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4eeb89-a725-4965-a5d7-ea9f361d8673 None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:26.745381 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4eeb89-a725-4965-a5d7-ea9f361d8673 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:26.745642 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4eeb89-a725-4965-a5d7-ea9f361d8673 None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:26.747130 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 495/992] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 30 15:47:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 15:47:26.752233 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409dd0dc-d41b-4676-9d79-cd28a7955bc6 None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:26.752524 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409dd0dc-d41b-4676-9d79-cd28a7955bc6 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:26.752770 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409dd0dc-d41b-4676-9d79-cd28a7955bc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:26.770970 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-409dd0dc-d41b-4676-9d79-cd28a7955bc6 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:47:26.772153 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-409dd0dc-d41b-4676-9d79-cd28a7955bc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:47:26.832824 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4eeb89-a725-4965-a5d7-ea9f361d8673 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:26.841332 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 498/993] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 30 15:47:26 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 15:47:26.907737 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:47:26.944599 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4eeb89-a725-4965-a5d7-ea9f361d8673 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:26.980446 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c17130f-6663-4194-ab20-debcdc06ec6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:26.981075 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c17130f-6663-4194-ab20-debcdc06ec6e None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:26.981376 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c17130f-6663-4194-ab20-debcdc06ec6e None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:26.981645 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c17130f-6663-4194-ab20-debcdc06ec6e None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:26.982244 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c17130f-6663-4194-ab20-debcdc06ec6e None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:26.983647 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c17130f-6663-4194-ab20-debcdc06ec6e None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:26.992529 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-9c17130f-6663-4194-ab20-debcdc06ec6e None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 30 15:47:26.992529 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:47:26.992529 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 15:47:26.992529 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 15:47:26.992529 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 15:47:26.992529 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 15:47:26.992529 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 15:47:26.992529 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 15:47:26.992529 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 30 15:47:26.992529 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Aug 30 15:47:26.992529 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 15:47:26.992529 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Aug 30 15:47:26.992529 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:47:26.992529 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:47:26.992529 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:47:26.992529 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:47:26.992529 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:47:26.993587 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:47:26.993587 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:47:26.993587 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:47:26.993587 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:47:26.993587 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:47:26.993587 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:47:26.993587 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 15:47:26.993587 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 15:47:26.993587 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 15:47:26.993587 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 15:47:26.993587 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:47:26.993587 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:47:26.993587 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:47:26.993587 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:47:26.993587 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:47:26.993587 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:47:26.994527 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:47:26.994527 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Aug 30 15:47:26.994527 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:47:26.994527 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:47:26.994527 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:47:26.994527 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:47:26.994527 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:47:26.994527 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:47:26.994527 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:47:26.994527 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:47:26.994527 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 15:47:26.994527 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 15:47:26.994527 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 30 15:47:26.994527 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Aug 30 15:47:26.994527 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 496/994] 10.222.0.20 () {56 vars in 1126 bytes} [Tue Aug 30 15:47:26 2022] GET /identity/v3/services/object-store => generated 93 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 15:47:26.999970 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409dd0dc-d41b-4676-9d79-cd28a7955bc6 None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:27.053775 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:47:27.105034 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409dd0dc-d41b-4676-9d79-cd28a7955bc6 None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:27.106360 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c17130f-6663-4194-ab20-debcdc06ec6e None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:27.142407 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167aa375-14b4-493e-99cd-fc195032ee84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:27.143500 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167aa375-14b4-493e-99cd-fc195032ee84 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:27.143500 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167aa375-14b4-493e-99cd-fc195032ee84 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:27.143500 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167aa375-14b4-493e-99cd-fc195032ee84 None admin] PATH_INFO: `/v3/services` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:27.144140 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167aa375-14b4-493e-99cd-fc195032ee84 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81512) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:47:27.144440 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167aa375-14b4-493e-99cd-fc195032ee84 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:27.145840 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167aa375-14b4-493e-99cd-fc195032ee84 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:27.150876 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 499/995] 10.222.0.20 () {56 vars in 1122 bytes} [Tue Aug 30 15:47:26 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:47:27.156396 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167aa375-14b4-493e-99cd-fc195032ee84 None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:27.210518 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c17130f-6663-4194-ab20-debcdc06ec6e None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:27.241881 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:47:27.248189 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f652a3-e473-4f56-a7da-f4f8bfbbc8c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:27.249009 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f652a3-e473-4f56-a7da-f4f8bfbbc8c0 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:27.249317 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f652a3-e473-4f56-a7da-f4f8bfbbc8c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:27.249587 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f652a3-e473-4f56-a7da-f4f8bfbbc8c0 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:27.250231 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f652a3-e473-4f56-a7da-f4f8bfbbc8c0 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:27.251543 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f652a3-e473-4f56-a7da-f4f8bfbbc8c0 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:27.260793 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-d5f652a3-e473-4f56-a7da-f4f8bfbbc8c0 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 30 15:47:27.260793 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:47:27.260793 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 15:47:27.260793 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 15:47:27.260793 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 15:47:27.260793 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 15:47:27.260793 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 15:47:27.260793 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 15:47:27.260793 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 30 15:47:27.260793 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Aug 30 15:47:27.260793 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 15:47:27.260793 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Aug 30 15:47:27.260793 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:47:27.260793 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:47:27.260793 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:47:27.260793 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:47:27.260793 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:47:27.262025 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:47:27.262025 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:47:27.262025 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:47:27.262025 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:47:27.262025 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:47:27.262025 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:47:27.262025 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 15:47:27.262025 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 15:47:27.262025 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 15:47:27.262025 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 15:47:27.262025 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:47:27.262025 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:47:27.262025 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:47:27.262025 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:47:27.262025 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:47:27.262025 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:47:27.262876 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:47:27.262876 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Aug 30 15:47:27.262876 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:47:27.262876 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:47:27.262876 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:47:27.262876 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:47:27.262876 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:47:27.262876 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:47:27.262876 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:47:27.262876 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:47:27.262876 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 15:47:27.262876 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 15:47:27.262876 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 30 15:47:27.262876 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Aug 30 15:47:27.262876 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 497/996] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Aug 30 15:47:27 2022] GET /identity/v3/services/compute => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 15:47:27.264922 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167aa375-14b4-493e-99cd-fc195032ee84 None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:27.267712 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f652a3-e473-4f56-a7da-f4f8bfbbc8c0 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:27.282164 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:47:27.306328 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003cbc47-7ba7-4680-a66e-67df3f875787 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:27.306529 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003cbc47-7ba7-4680-a66e-67df3f875787 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:27.306788 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003cbc47-7ba7-4680-a66e-67df3f875787 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:27.307027 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003cbc47-7ba7-4680-a66e-67df3f875787 None admin] PATH_INFO: `/v3/services` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:27.307656 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003cbc47-7ba7-4680-a66e-67df3f875787 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81512) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:47:27.307963 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003cbc47-7ba7-4680-a66e-67df3f875787 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:27.309364 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003cbc47-7ba7-4680-a66e-67df3f875787 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:27.319187 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 500/997] 10.222.0.20 () {56 vars in 1122 bytes} [Tue Aug 30 15:47:27 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 15:47:27.320347 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003cbc47-7ba7-4680-a66e-67df3f875787 None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:27.374309 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f652a3-e473-4f56-a7da-f4f8bfbbc8c0 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:27.414853 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a964773-c69d-43fd-8ea3-8c43b8fd799c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:27.415464 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a964773-c69d-43fd-8ea3-8c43b8fd799c None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:27.415739 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a964773-c69d-43fd-8ea3-8c43b8fd799c None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:27.416004 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a964773-c69d-43fd-8ea3-8c43b8fd799c None admin] PATH_INFO: `/v3/services` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:27.416636 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a964773-c69d-43fd-8ea3-8c43b8fd799c None admin] RBAC: Adding query filter params (name=compute) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:47:27.416933 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a964773-c69d-43fd-8ea3-8c43b8fd799c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:27.418343 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a964773-c69d-43fd-8ea3-8c43b8fd799c None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:27.423769 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 498/998] 10.222.0.20 () {56 vars in 1112 bytes} [Tue Aug 30 15:47:27 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:47:27.429242 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a964773-c69d-43fd-8ea3-8c43b8fd799c None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:27.430022 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003cbc47-7ba7-4680-a66e-67df3f875787 None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:27.467083 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6fcb42-d367-4546-88a9-7494138b13b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:27.467736 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6fcb42-d367-4546-88a9-7494138b13b7 None admin] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:27.468017 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6fcb42-d367-4546-88a9-7494138b13b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:27.468226 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6fcb42-d367-4546-88a9-7494138b13b7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:27.468893 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6fcb42-d367-4546-88a9-7494138b13b7 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:27.470301 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6fcb42-d367-4546-88a9-7494138b13b7 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:27.487571 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 501/999] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Aug 30 15:47:27 2022] POST /identity/v3/endpoints => generated 325 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 15:47:27.490750 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:47:27.539798 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a964773-c69d-43fd-8ea3-8c43b8fd799c None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:27.582847 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2eac7d-c733-42f5-ba10-ac04d20ad39b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:27.583467 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2eac7d-c733-42f5-ba10-ac04d20ad39b None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:27.583742 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2eac7d-c733-42f5-ba10-ac04d20ad39b None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:27.584008 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2eac7d-c733-42f5-ba10-ac04d20ad39b None admin] PATH_INFO: `/v3/services` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:27.584648 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2eac7d-c733-42f5-ba10-ac04d20ad39b None admin] RBAC: Adding query filter params (type=compute) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:47:27.584947 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2eac7d-c733-42f5-ba10-ac04d20ad39b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:27.586375 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2eac7d-c733-42f5-ba10-ac04d20ad39b None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:27.592041 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 499/1000] 10.222.0.20 () {56 vars in 1112 bytes} [Tue Aug 30 15:47:27 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 15:47:27.597953 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6fcb42-d367-4546-88a9-7494138b13b7 None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:27.648982 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:47:27.714821 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6fcb42-d367-4546-88a9-7494138b13b7 None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:27.749119 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c0042a-4cdb-4cb1-bfa4-b0b2b30c44f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:27.749757 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c0042a-4cdb-4cb1-bfa4-b0b2b30c44f7 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:27.750030 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c0042a-4cdb-4cb1-bfa4-b0b2b30c44f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:27.750294 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c0042a-4cdb-4cb1-bfa4-b0b2b30c44f7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:27.750999 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c0042a-4cdb-4cb1-bfa4-b0b2b30c44f7 None admin] RBAC: Adding query filter params (interface=public, service_id=a25e654ea31d4564b286354b20be288e, region_id=RegionOne) {{(pid=81512) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:47:27.751301 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c0042a-4cdb-4cb1-bfa4-b0b2b30c44f7 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:27.752667 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c0042a-4cdb-4cb1-bfa4-b0b2b30c44f7 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:27.758078 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 502/1001] 10.222.0.20 () {56 vars in 1251 bytes} [Tue Aug 30 15:47:27 2022] GET /identity/v3/endpoints?service_id=a25e654ea31d4564b286354b20be288e&interface=public®ion_id=RegionOne => generated 194 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:47:27.763539 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2eac7d-c733-42f5-ba10-ac04d20ad39b None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:27.845071 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde9903c-8d7c-4dff-bad9-7489d6c8352b None None] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:27.845412 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde9903c-8d7c-4dff-bad9-7489d6c8352b None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:27.845718 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde9903c-8d7c-4dff-bad9-7489d6c8352b None None] PATH_INFO: `/` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:27.847122 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 503/1002] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 30 15:47:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 15:47:27.852262 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a5592b-4cc2-46d9-8764-6e7a9d656ac9 None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:27.852537 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a5592b-4cc2-46d9-8764-6e7a9d656ac9 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:27.852818 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a5592b-4cc2-46d9-8764-6e7a9d656ac9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:27.867395 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2eac7d-c733-42f5-ba10-ac04d20ad39b None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:27.873690 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-05a5592b-4cc2-46d9-8764-6e7a9d656ac9 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:47:27.875259 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-05a5592b-4cc2-46d9-8764-6e7a9d656ac9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:47:27.887364 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:47:27.903698 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fc80d8-d1d3-42c4-9a40-9e739fc1d393 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:27.904401 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fc80d8-d1d3-42c4-9a40-9e739fc1d393 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:27.904681 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fc80d8-d1d3-42c4-9a40-9e739fc1d393 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:27.904948 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fc80d8-d1d3-42c4-9a40-9e739fc1d393 None admin] PATH_INFO: `/v3/services` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:27.905590 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fc80d8-d1d3-42c4-9a40-9e739fc1d393 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:27.906965 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fc80d8-d1d3-42c4-9a40-9e739fc1d393 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:27.913395 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 500/1003] 10.222.0.20 () {56 vars in 1087 bytes} [Tue Aug 30 15:47:27 2022] GET /identity/v3/services => generated 1727 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 15:47:27.959152 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 504/1004] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 30 15:47:27 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 30 15:47:28.226766 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fc80d8-d1d3-42c4-9a40-9e739fc1d393 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:28.329973 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fc80d8-d1d3-42c4-9a40-9e739fc1d393 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:28.364843 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284560d4-7ce8-44ee-aadd-c8fb95d16f5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:28.365874 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284560d4-7ce8-44ee-aadd-c8fb95d16f5c None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:28.366362 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284560d4-7ce8-44ee-aadd-c8fb95d16f5c None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:28.366841 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284560d4-7ce8-44ee-aadd-c8fb95d16f5c None admin] PATH_INFO: `/v3/services/image` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:28.367621 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284560d4-7ce8-44ee-aadd-c8fb95d16f5c None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:28.369167 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284560d4-7ce8-44ee-aadd-c8fb95d16f5c None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:28.379101 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-284560d4-7ce8-44ee-aadd-c8fb95d16f5c None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 30 15:47:28.379101 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:47:28.379101 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 15:47:28.379101 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 15:47:28.379101 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 15:47:28.379101 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 15:47:28.379101 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 15:47:28.379101 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 15:47:28.379101 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 30 15:47:28.379101 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Aug 30 15:47:28.379101 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 15:47:28.379101 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Aug 30 15:47:28.379101 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:47:28.379101 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:47:28.379101 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:47:28.379101 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:47:28.379101 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:47:28.380181 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:47:28.380181 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:47:28.380181 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:47:28.380181 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:47:28.380181 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:47:28.380181 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:47:28.380181 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 15:47:28.380181 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 15:47:28.380181 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 15:47:28.380181 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 15:47:28.380181 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:47:28.380181 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:47:28.380181 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:47:28.380181 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:47:28.380181 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:47:28.380181 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:47:28.381727 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:47:28.381727 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Aug 30 15:47:28.381727 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:47:28.381727 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:47:28.381727 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:47:28.381727 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:47:28.381727 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:47:28.381727 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:47:28.381727 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:47:28.381727 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:47:28.381727 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 15:47:28.381727 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 15:47:28.381727 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 30 15:47:28.381727 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Aug 30 15:47:28.381727 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 501/1005] 10.222.0.20 () {56 vars in 1105 bytes} [Tue Aug 30 15:47:28 2022] GET /identity/v3/services/image => generated 86 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 15:47:28.387121 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a5592b-4cc2-46d9-8764-6e7a9d656ac9 None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:28.424430 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:47:28.498541 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a5592b-4cc2-46d9-8764-6e7a9d656ac9 None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:28.532875 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebf0639-6202-424c-9f8d-266c90deca7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:28.533550 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebf0639-6202-424c-9f8d-266c90deca7b None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:28.533827 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebf0639-6202-424c-9f8d-266c90deca7b None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:28.534097 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebf0639-6202-424c-9f8d-266c90deca7b None admin] PATH_INFO: `/v3/services` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:28.534710 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebf0639-6202-424c-9f8d-266c90deca7b None admin] RBAC: Adding query filter params (name=image) {{(pid=81512) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:47:28.535012 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebf0639-6202-424c-9f8d-266c90deca7b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:28.536392 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebf0639-6202-424c-9f8d-266c90deca7b None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:28.541534 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 505/1006] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 30 15:47:28 2022] GET /identity/v3/services?name=image => generated 122 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:47:28.547071 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284560d4-7ce8-44ee-aadd-c8fb95d16f5c None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:28.600621 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:47:28.661196 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284560d4-7ce8-44ee-aadd-c8fb95d16f5c None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:28.695401 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d0164a-1d26-462f-8e9b-ac23ac4a926f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:28.696022 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d0164a-1d26-462f-8e9b-ac23ac4a926f None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:28.696274 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d0164a-1d26-462f-8e9b-ac23ac4a926f None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:28.696562 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d0164a-1d26-462f-8e9b-ac23ac4a926f None admin] PATH_INFO: `/v3/services` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:28.697173 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d0164a-1d26-462f-8e9b-ac23ac4a926f None admin] RBAC: Adding query filter params (type=image) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:47:28.697493 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d0164a-1d26-462f-8e9b-ac23ac4a926f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:28.698865 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d0164a-1d26-462f-8e9b-ac23ac4a926f None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:28.703526 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 502/1007] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 30 15:47:28 2022] GET /identity/v3/services?type=image => generated 122 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:47:29.415589 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9471efed-710d-4f7a-83a5-6e481c6f2a14 None None] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:29.416026 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9471efed-710d-4f7a-83a5-6e481c6f2a14 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:29.416147 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9471efed-710d-4f7a-83a5-6e481c6f2a14 None None] PATH_INFO: `/` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:29.417677 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 506/1008] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 30 15:47:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 15:47:29.422837 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518d4e9c-a443-4e63-a5e5-fea332921108 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:29.423099 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518d4e9c-a443-4e63-a5e5-fea332921108 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:29.423401 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518d4e9c-a443-4e63-a5e5-fea332921108 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:29.442339 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-518d4e9c-a443-4e63-a5e5-fea332921108 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:47:29.443532 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-518d4e9c-a443-4e63-a5e5-fea332921108 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:47:29.511522 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 503/1009] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 30 15:47:29 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 30 15:47:29.770313 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9471efed-710d-4f7a-83a5-6e481c6f2a14 None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:29.812220 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1476aca4-66b9-4f52-b198-d0d7763c5962 None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:29.812742 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1476aca4-66b9-4f52-b198-d0d7763c5962 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:29.812742 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1476aca4-66b9-4f52-b198-d0d7763c5962 None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:29.814255 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 504/1010] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 30 15:47:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 15:47:29.818959 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baee2dd3-ac69-4332-b45e-9dcb7f872249 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:29.819277 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baee2dd3-ac69-4332-b45e-9dcb7f872249 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:29.819588 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baee2dd3-ac69-4332-b45e-9dcb7f872249 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:29.837831 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-baee2dd3-ac69-4332-b45e-9dcb7f872249 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:47:29.839028 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-baee2dd3-ac69-4332-b45e-9dcb7f872249 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:47:29.873115 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:47:29.878988 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:47:29.882405 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9471efed-710d-4f7a-83a5-6e481c6f2a14 None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:29.911189 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 505/1011] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 30 15:47:29 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 30 15:47:29.920156 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb3763d-1d17-47fd-9f73-448803de6a35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:29.920869 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb3763d-1d17-47fd-9f73-448803de6a35 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:29.921229 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb3763d-1d17-47fd-9f73-448803de6a35 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:29.921574 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb3763d-1d17-47fd-9f73-448803de6a35 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:29.922753 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb3763d-1d17-47fd-9f73-448803de6a35 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:29.924252 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb3763d-1d17-47fd-9f73-448803de6a35 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:29.926044 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 507/1012] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 30 15:47:29 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:47:29.931638 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baee2dd3-ac69-4332-b45e-9dcb7f872249 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:30.036251 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baee2dd3-ac69-4332-b45e-9dcb7f872249 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:30.073153 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288aa585-6b74-4d5a-a43b-13fec2037c63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:30.073792 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288aa585-6b74-4d5a-a43b-13fec2037c63 None admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:30.074064 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288aa585-6b74-4d5a-a43b-13fec2037c63 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:30.074333 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288aa585-6b74-4d5a-a43b-13fec2037c63 None admin] PATH_INFO: `/v3/projects` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:30.074986 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288aa585-6b74-4d5a-a43b-13fec2037c63 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:30.076459 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288aa585-6b74-4d5a-a43b-13fec2037c63 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:30.105297 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 506/1013] 10.222.0.20 () {60 vars in 1141 bytes} [Tue Aug 30 15:47:29 2022] POST /identity/v3/projects => generated 311 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 15:47:30.185824 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb3763d-1d17-47fd-9f73-448803de6a35 None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:30.298941 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb3763d-1d17-47fd-9f73-448803de6a35 None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:30.333184 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b29f3eb-0249-450c-bbee-cd601ba2e188 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:30.333843 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b29f3eb-0249-450c-bbee-cd601ba2e188 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:30.334116 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b29f3eb-0249-450c-bbee-cd601ba2e188 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:30.334381 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b29f3eb-0249-450c-bbee-cd601ba2e188 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:30.334967 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b29f3eb-0249-450c-bbee-cd601ba2e188 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:30.336333 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b29f3eb-0249-450c-bbee-cd601ba2e188 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:30.345576 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application [None req-4b29f3eb-0249-450c-bbee-cd601ba2e188 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 30 15:47:30.345576 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:47:30.345576 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 15:47:30.345576 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 15:47:30.345576 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 15:47:30.345576 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 15:47:30.345576 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 15:47:30.345576 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 15:47:30.345576 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 30 15:47:30.345576 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Aug 30 15:47:30.345576 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 15:47:30.345576 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Aug 30 15:47:30.345576 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:47:30.345576 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:47:30.345576 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:47:30.345576 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:47:30.345576 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:47:30.346634 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:47:30.346634 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:47:30.346634 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:47:30.346634 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:47:30.346634 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:47:30.346634 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:47:30.346634 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 15:47:30.346634 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 15:47:30.346634 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 15:47:30.346634 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 15:47:30.346634 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:47:30.346634 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:47:30.346634 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:47:30.346634 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:47:30.346634 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:47:30.346634 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:47:30.347524 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:47:30.347524 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application with Lock( Aug 30 15:47:30.347524 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:47:30.347524 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:47:30.347524 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:47:30.347524 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:47:30.347524 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:47:30.347524 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:47:30.347524 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:47:30.347524 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:47:30.347524 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 15:47:30.347524 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 15:47:30.347524 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 30 15:47:30.347524 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Aug 30 15:47:30.347524 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 508/1014] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Aug 30 15:47:30 2022] GET /identity/v3/services/compute => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 15:47:30.354814 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288aa585-6b74-4d5a-a43b-13fec2037c63 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:30.382445 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:47:30.465548 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288aa585-6b74-4d5a-a43b-13fec2037c63 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:30.503915 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbdca96-e302-4e43-902b-aa4f66d214ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:30.504529 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbdca96-e302-4e43-902b-aa4f66d214ae None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:30.504807 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbdca96-e302-4e43-902b-aa4f66d214ae None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:30.505074 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbdca96-e302-4e43-902b-aa4f66d214ae None admin] PATH_INFO: `/v3/services` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:30.505742 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbdca96-e302-4e43-902b-aa4f66d214ae None admin] RBAC: Adding query filter params (name=compute) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:47:30.506042 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbdca96-e302-4e43-902b-aa4f66d214ae None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:30.507419 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbdca96-e302-4e43-902b-aa4f66d214ae None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:30.517287 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 507/1015] 10.222.0.20 () {56 vars in 1112 bytes} [Tue Aug 30 15:47:30 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:47:30.518410 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b29f3eb-0249-450c-bbee-cd601ba2e188 None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:30.548730 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:47:30.577428 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda07b85-d1c6-43f3-bf44-80b4432ae8ec None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:30.577732 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda07b85-d1c6-43f3-bf44-80b4432ae8ec None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:30.577900 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda07b85-d1c6-43f3-bf44-80b4432ae8ec None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:30.579400 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 508/1016] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 30 15:47:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 15:47:30.584234 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2238228-2139-4c4b-9369-30696ae2f8aa None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:30.584521 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2238228-2139-4c4b-9369-30696ae2f8aa None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:30.584796 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2238228-2139-4c4b-9369-30696ae2f8aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:30.604965 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-f2238228-2139-4c4b-9369-30696ae2f8aa None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:47:30.606219 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-f2238228-2139-4c4b-9369-30696ae2f8aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:47:30.631685 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b29f3eb-0249-450c-bbee-cd601ba2e188 None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:30.667596 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d151a1-daa4-42fd-b020-0c4f66a1791b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:30.668216 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d151a1-daa4-42fd-b020-0c4f66a1791b None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:30.668494 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d151a1-daa4-42fd-b020-0c4f66a1791b None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:30.668761 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d151a1-daa4-42fd-b020-0c4f66a1791b None admin] PATH_INFO: `/v3/services` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:30.669421 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d151a1-daa4-42fd-b020-0c4f66a1791b None admin] RBAC: Adding query filter params (type=compute) {{(pid=81512) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:47:30.669753 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d151a1-daa4-42fd-b020-0c4f66a1791b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:30.671128 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d151a1-daa4-42fd-b020-0c4f66a1791b None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:30.680455 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 509/1017] 10.222.0.20 () {56 vars in 1112 bytes} [Tue Aug 30 15:47:30 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:47:30.681450 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d151a1-daa4-42fd-b020-0c4f66a1791b None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:30.690452 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 509/1018] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 30 15:47:30 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 30 15:47:30.774716 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:47:30.785681 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d151a1-daa4-42fd-b020-0c4f66a1791b None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:30.819115 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517af8d4-a816-41e6-9e02-3d866ca1c148 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:30.819718 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517af8d4-a816-41e6-9e02-3d866ca1c148 None admin] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:30.820019 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517af8d4-a816-41e6-9e02-3d866ca1c148 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:30.820291 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517af8d4-a816-41e6-9e02-3d866ca1c148 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:30.820949 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517af8d4-a816-41e6-9e02-3d866ca1c148 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:30.822358 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517af8d4-a816-41e6-9e02-3d866ca1c148 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:30.844055 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 510/1019] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Aug 30 15:47:30 2022] POST /identity/v3/endpoints => generated 334 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 15:47:30.954712 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2238228-2139-4c4b-9369-30696ae2f8aa None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:30.974753 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:47:31.064744 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2238228-2139-4c4b-9369-30696ae2f8aa None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:31.106834 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1c5f06-fa8e-4c7b-a9d8-12f90fbe764a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:31.107432 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1c5f06-fa8e-4c7b-a9d8-12f90fbe764a None admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:31.107707 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1c5f06-fa8e-4c7b-a9d8-12f90fbe764a None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:31.107976 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1c5f06-fa8e-4c7b-a9d8-12f90fbe764a None admin] PATH_INFO: `/v3/services` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:31.108644 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1c5f06-fa8e-4c7b-a9d8-12f90fbe764a None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:31.110055 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1c5f06-fa8e-4c7b-a9d8-12f90fbe764a None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:31.127043 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 510/1020] 10.222.0.20 () {60 vars in 1141 bytes} [Tue Aug 30 15:47:30 2022] POST /identity/v3/services => generated 242 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 15:47:32.035065 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfac8d6-b980-4c3c-a059-efee39cab972 None None] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:32.035606 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfac8d6-b980-4c3c-a059-efee39cab972 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:32.035606 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfac8d6-b980-4c3c-a059-efee39cab972 None None] PATH_INFO: `/` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:32.037265 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 511/1021] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 30 15:47:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 15:47:32.042248 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4500c08a-5a31-4072-89d2-bdadeb703a81 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:32.042994 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4500c08a-5a31-4072-89d2-bdadeb703a81 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:32.043457 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4500c08a-5a31-4072-89d2-bdadeb703a81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:32.063102 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-4500c08a-5a31-4072-89d2-bdadeb703a81 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:47:32.064540 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-4500c08a-5a31-4072-89d2-bdadeb703a81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:47:32.181969 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:47:32.189282 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 511/1022] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 30 15:47:32 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 15:47:32.449249 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfac8d6-b980-4c3c-a059-efee39cab972 None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:32.555690 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfac8d6-b980-4c3c-a059-efee39cab972 None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:32.591195 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e216eb2-ee57-4a62-bea9-d06ee685fe03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:32.591861 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e216eb2-ee57-4a62-bea9-d06ee685fe03 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:32.592141 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e216eb2-ee57-4a62-bea9-d06ee685fe03 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:32.592436 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e216eb2-ee57-4a62-bea9-d06ee685fe03 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:32.593575 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e216eb2-ee57-4a62-bea9-d06ee685fe03 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:32.595091 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e216eb2-ee57-4a62-bea9-d06ee685fe03 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:32.596852 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 512/1023] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 30 15:47:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:47:32.602614 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4500c08a-5a31-4072-89d2-bdadeb703a81 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:32.715093 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4500c08a-5a31-4072-89d2-bdadeb703a81 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:32.749381 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2c25ae-9549-45b9-ba4d-2c80d15cbb8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:32.749992 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2c25ae-9549-45b9-ba4d-2c80d15cbb8a None admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:32.750266 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2c25ae-9549-45b9-ba4d-2c80d15cbb8a None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:32.750535 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2c25ae-9549-45b9-ba4d-2c80d15cbb8a None admin] PATH_INFO: `/v3/users` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:32.751201 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2c25ae-9549-45b9-ba4d-2c80d15cbb8a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:32.752647 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2c25ae-9549-45b9-ba4d-2c80d15cbb8a None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:32.777598 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 512/1024] 10.222.0.20 () {60 vars in 1132 bytes} [Tue Aug 30 15:47:32 2022] POST /identity/v3/users => generated 285 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 15:47:32.835719 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f257d7-8156-4b92-b713-c0dbda089f9e None None] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:32.836087 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f257d7-8156-4b92-b713-c0dbda089f9e None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:32.836231 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f257d7-8156-4b92-b713-c0dbda089f9e None None] PATH_INFO: `/` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:32.837923 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 513/1025] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 30 15:47:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 15:47:32.843815 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab2c456-c4fb-41d5-9c39-0a92b6ef2df6 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:32.844112 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab2c456-c4fb-41d5-9c39-0a92b6ef2df6 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:32.844364 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab2c456-c4fb-41d5-9c39-0a92b6ef2df6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:32.864513 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-dab2c456-c4fb-41d5-9c39-0a92b6ef2df6 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:47:32.865770 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-dab2c456-c4fb-41d5-9c39-0a92b6ef2df6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:47:32.924364 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:47:32.945071 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 513/1026] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 30 15:47:32 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 15:47:33.084405 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fb7e3c-3a91-4670-819d-dd9f49b8d2bf None None] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:33.084914 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fb7e3c-3a91-4670-819d-dd9f49b8d2bf None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:33.084914 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fb7e3c-3a91-4670-819d-dd9f49b8d2bf None None] PATH_INFO: `/` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:33.086301 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 514/1027] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 30 15:47:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 15:47:33.091869 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881ce64d-272b-420f-934e-6942190fb632 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:33.092308 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881ce64d-272b-420f-934e-6942190fb632 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:33.092385 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881ce64d-272b-420f-934e-6942190fb632 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:33.111023 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-881ce64d-272b-420f-934e-6942190fb632 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:47:33.112186 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-881ce64d-272b-420f-934e-6942190fb632 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:47:33.183380 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 514/1028] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 30 15:47:33 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 15:47:33.208921 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fb7e3c-3a91-4670-819d-dd9f49b8d2bf None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:33.227594 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:47:33.316893 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fb7e3c-3a91-4670-819d-dd9f49b8d2bf None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:33.351030 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c310d063-cab3-43ba-ba40-8350e436131c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:33.351580 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c310d063-cab3-43ba-ba40-8350e436131c None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:33.351855 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c310d063-cab3-43ba-ba40-8350e436131c None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:33.352125 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c310d063-cab3-43ba-ba40-8350e436131c None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:33.362333 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c310d063-cab3-43ba-ba40-8350e436131c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 15:47:33.367132 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application [None req-c310d063-cab3-43ba-ba40-8350e436131c None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 30 15:47:33.367132 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:47:33.367132 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:47:33.367132 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:47:33.367132 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:47:33.367132 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:47:33.367132 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:47:33.367132 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:47:33.367132 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:47:33.367132 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:47:33.367132 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:47:33.367132 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:47:33.367132 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 15:47:33.367132 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 15:47:33.367132 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 15:47:33.367132 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 15:47:33.368049 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 15:47:33.368049 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 15:47:33.368049 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 15:47:33.368049 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 15:47:33.368049 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:47:33.368049 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:47:33.368049 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:47:33.368049 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:47:33.368049 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:47:33.368049 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:47:33.368049 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:47:33.368049 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application with Lock( Aug 30 15:47:33.368049 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:47:33.368049 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:47:33.368049 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:47:33.368049 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:47:33.368924 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:47:33.368924 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:47:33.368924 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:47:33.368924 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:47:33.368924 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 15:47:33.368924 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 15:47:33.368924 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 15:47:33.368924 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 15:47:33.368924 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 15:47:33.368924 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 15:47:33.368924 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 30 15:47:33.368924 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Aug 30 15:47:33.368924 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 515/1029] 10.222.0.20 () {56 vars in 1120 bytes} [Tue Aug 30 15:47:33 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 15:47:33.374616 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881ce64d-272b-420f-934e-6942190fb632 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:33.448974 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c310d063-cab3-43ba-ba40-8350e436131c None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:33.473284 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881ce64d-272b-420f-934e-6942190fb632 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:33.492877 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:47:33.514066 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72289380-4f72-4956-8b11-c0edea0fc564 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:33.514690 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72289380-4f72-4956-8b11-c0edea0fc564 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:33.514967 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72289380-4f72-4956-8b11-c0edea0fc564 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:33.515241 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72289380-4f72-4956-8b11-c0edea0fc564 None admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:33.515912 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72289380-4f72-4956-8b11-c0edea0fc564 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:47:33.516207 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72289380-4f72-4956-8b11-c0edea0fc564 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:33.517626 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72289380-4f72-4956-8b11-c0edea0fc564 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:33.525963 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 515/1030] 10.222.0.20 () {56 vars in 1115 bytes} [Tue Aug 30 15:47:33 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:47:33.531654 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72289380-4f72-4956-8b11-c0edea0fc564 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:33.533769 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:47:33.560202 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c310d063-cab3-43ba-ba40-8350e436131c None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:33.602550 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a4e88e-c376-4d31-9885-1f5ca206dc6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:33.602758 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a4e88e-c376-4d31-9885-1f5ca206dc6d None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:33.602951 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a4e88e-c376-4d31-9885-1f5ca206dc6d None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:33.603296 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a4e88e-c376-4d31-9885-1f5ca206dc6d None admin] PATH_INFO: `/v3/services/image` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:33.603888 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a4e88e-c376-4d31-9885-1f5ca206dc6d None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:33.605297 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a4e88e-c376-4d31-9885-1f5ca206dc6d None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:33.614428 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application [None req-c3a4e88e-c376-4d31-9885-1f5ca206dc6d None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 30 15:47:33.614428 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:47:33.614428 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 15:47:33.614428 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 15:47:33.614428 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 15:47:33.614428 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 15:47:33.614428 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 15:47:33.614428 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 15:47:33.614428 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 30 15:47:33.614428 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Aug 30 15:47:33.614428 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 15:47:33.614428 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Aug 30 15:47:33.614428 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:47:33.614428 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:47:33.614428 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:47:33.614428 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:47:33.614428 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:47:33.615421 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:47:33.615421 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:47:33.615421 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:47:33.615421 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:47:33.615421 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:47:33.615421 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:47:33.615421 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 15:47:33.615421 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 15:47:33.615421 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 15:47:33.615421 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 15:47:33.615421 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:47:33.615421 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:47:33.615421 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:47:33.615421 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:47:33.615421 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:47:33.615421 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:47:33.616412 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:47:33.616412 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application with Lock( Aug 30 15:47:33.616412 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:47:33.616412 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:47:33.616412 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:47:33.616412 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:47:33.616412 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:47:33.616412 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:47:33.616412 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:47:33.616412 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:47:33.616412 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 15:47:33.616412 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 15:47:33.616412 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 30 15:47:33.616412 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Aug 30 15:47:33.616412 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 516/1031] 10.222.0.20 () {56 vars in 1105 bytes} [Tue Aug 30 15:47:33 2022] GET /identity/v3/services/image => generated 86 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 15:47:33.621756 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a4e88e-c376-4d31-9885-1f5ca206dc6d None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:33.642414 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72289380-4f72-4956-8b11-c0edea0fc564 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:33.678369 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0c4556-8a6a-459a-8918-9ca7473bdfe8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:33.679011 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0c4556-8a6a-459a-8918-9ca7473bdfe8 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:33.679332 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0c4556-8a6a-459a-8918-9ca7473bdfe8 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:33.679607 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0c4556-8a6a-459a-8918-9ca7473bdfe8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:33.690978 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0c4556-8a6a-459a-8918-9ca7473bdfe8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:33.692433 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0c4556-8a6a-459a-8918-9ca7473bdfe8 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:33.706776 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-ad0c4556-8a6a-459a-8918-9ca7473bdfe8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 15:47:33.706776 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:47:33.706776 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:47:33.706776 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:47:33.706776 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:47:33.706776 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:47:33.706776 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:47:33.706776 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:47:33.706776 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:47:33.706776 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:47:33.706776 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:47:33.706776 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:47:33.706776 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 15:47:33.706776 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 15:47:33.706776 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 15:47:33.706776 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 15:47:33.707762 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:47:33.707762 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:47:33.707762 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:47:33.707762 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:47:33.707762 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:47:33.707762 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:47:33.707762 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:47:33.707762 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Aug 30 15:47:33.707762 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:47:33.707762 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:47:33.707762 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:47:33.707762 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:47:33.707762 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:47:33.707762 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:47:33.707762 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:47:33.707762 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:47:33.707762 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 15:47:33.708694 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 15:47:33.708694 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 15:47:33.708694 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Aug 30 15:47:33.708694 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 516/1032] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 30 15:47:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 15:47:33.713848 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0c4556-8a6a-459a-8918-9ca7473bdfe8 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:33.729657 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a4e88e-c376-4d31-9885-1f5ca206dc6d None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:33.745788 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:47:33.767247 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e47c4a-61d8-4290-816c-571ae8f960ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:33.767857 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e47c4a-61d8-4290-816c-571ae8f960ab None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:33.768134 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e47c4a-61d8-4290-816c-571ae8f960ab None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:33.768401 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e47c4a-61d8-4290-816c-571ae8f960ab None admin] PATH_INFO: `/v3/services` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:33.769044 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e47c4a-61d8-4290-816c-571ae8f960ab None admin] RBAC: Adding query filter params (name=image) {{(pid=81512) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:47:33.769370 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e47c4a-61d8-4290-816c-571ae8f960ab None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:33.770744 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e47c4a-61d8-4290-816c-571ae8f960ab None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:33.775815 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 517/1033] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 30 15:47:33 2022] GET /identity/v3/services?name=image => generated 122 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 15:47:33.782018 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e47c4a-61d8-4290-816c-571ae8f960ab None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:33.785693 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:47:33.824626 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0c4556-8a6a-459a-8918-9ca7473bdfe8 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:33.863359 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2ba55b-f15a-4a16-9270-bfca876d17b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:33.863887 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2ba55b-f15a-4a16-9270-bfca876d17b4 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:33.864167 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2ba55b-f15a-4a16-9270-bfca876d17b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:33.864441 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2ba55b-f15a-4a16-9270-bfca876d17b4 None admin] PATH_INFO: `/v3/domains` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:33.865087 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2ba55b-f15a-4a16-9270-bfca876d17b4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:47:33.865488 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2ba55b-f15a-4a16-9270-bfca876d17b4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:33.866869 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2ba55b-f15a-4a16-9270-bfca876d17b4 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:33.879710 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 517/1034] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Aug 30 15:47:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:47:33.884989 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e47c4a-61d8-4290-816c-571ae8f960ab None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:33.885115 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2ba55b-f15a-4a16-9270-bfca876d17b4 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:33.920310 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ba54c9-49a0-47cf-9050-e825cd763def None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:33.920937 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ba54c9-49a0-47cf-9050-e825cd763def None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:33.921204 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ba54c9-49a0-47cf-9050-e825cd763def None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:33.922336 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ba54c9-49a0-47cf-9050-e825cd763def None admin] PATH_INFO: `/v3/services` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:33.922336 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ba54c9-49a0-47cf-9050-e825cd763def None admin] RBAC: Adding query filter params (type=image) {{(pid=81512) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:47:33.922555 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ba54c9-49a0-47cf-9050-e825cd763def None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:33.923795 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ba54c9-49a0-47cf-9050-e825cd763def None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:33.929851 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 518/1035] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 30 15:47:33 2022] GET /identity/v3/services?type=image => generated 350 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:47:33.936018 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ba54c9-49a0-47cf-9050-e825cd763def None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:33.984829 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:47:33.996510 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2ba55b-f15a-4a16-9270-bfca876d17b4 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:34.005698 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:47:34.037231 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d53945d-4620-4453-8060-44744f4f20ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:34.037878 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d53945d-4620-4453-8060-44744f4f20ff None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:34.038152 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d53945d-4620-4453-8060-44744f4f20ff None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:34.038406 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d53945d-4620-4453-8060-44744f4f20ff None admin] PATH_INFO: `/v3/users/nova` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:34.041181 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ba54c9-49a0-47cf-9050-e825cd763def None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:34.045830 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d53945d-4620-4453-8060-44744f4f20ff None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:34.047281 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d53945d-4620-4453-8060-44744f4f20ff None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:34.058808 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-2d53945d-4620-4453-8060-44744f4f20ff None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 30 15:47:34.058808 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:47:34.058808 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:47:34.058808 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:47:34.058808 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:47:34.058808 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:47:34.058808 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:47:34.058808 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:47:34.058808 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:47:34.058808 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:47:34.058808 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:47:34.058808 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:47:34.058808 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 15:47:34.058808 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 15:47:34.058808 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 15:47:34.058808 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 15:47:34.059789 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:47:34.059789 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:47:34.059789 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 15:47:34.059789 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 15:47:34.059789 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 15:47:34.059789 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 15:47:34.059789 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:47:34.059789 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:47:34.059789 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:47:34.059789 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:47:34.059789 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:47:34.059789 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Aug 30 15:47:34.059789 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:47:34.059789 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:47:34.059789 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:47:34.059789 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:47:34.059789 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:47:34.059789 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:47:34.066248 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:47:34.066248 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:47:34.066248 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 15:47:34.066248 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 15:47:34.066248 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 15:47:34.066248 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 15:47:34.066248 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 15:47:34.066248 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 15:47:34.066248 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 30 15:47:34.066248 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Aug 30 15:47:34.066248 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 518/1036] 10.222.0.20 () {56 vars in 1093 bytes} [Tue Aug 30 15:47:33 2022] GET /identity/v3/users/nova => generated 82 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 15:47:34.066248 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d53945d-4620-4453-8060-44744f4f20ff None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:34.079532 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddd55e7-c4e1-40f3-bf98-c239d3afaaad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:34.080137 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddd55e7-c4e1-40f3-bf98-c239d3afaaad None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:34.080417 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddd55e7-c4e1-40f3-bf98-c239d3afaaad None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:34.080676 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddd55e7-c4e1-40f3-bf98-c239d3afaaad None admin] PATH_INFO: `/v3/endpoints` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:34.081583 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddd55e7-c4e1-40f3-bf98-c239d3afaaad None admin] RBAC: Adding query filter params (interface=public, service_id=57b398fecb3b4e26a1d63e199f683b7b, region_id=RegionOne) {{(pid=81512) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:47:34.081895 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddd55e7-c4e1-40f3-bf98-c239d3afaaad None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:34.083238 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddd55e7-c4e1-40f3-bf98-c239d3afaaad None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:34.088667 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 519/1037] 10.222.0.20 () {56 vars in 1251 bytes} [Tue Aug 30 15:47:33 2022] GET /identity/v3/endpoints?service_id=57b398fecb3b4e26a1d63e199f683b7b&interface=public®ion_id=RegionOne => generated 194 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 15:47:34.097104 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddd55e7-c4e1-40f3-bf98-c239d3afaaad None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:34.188290 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d53945d-4620-4453-8060-44744f4f20ff None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:34.209698 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddd55e7-c4e1-40f3-bf98-c239d3afaaad None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:34.229662 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a90faf4-d03e-417d-85da-b18066d4ef01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:34.230280 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a90faf4-d03e-417d-85da-b18066d4ef01 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:34.230559 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a90faf4-d03e-417d-85da-b18066d4ef01 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:34.230828 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a90faf4-d03e-417d-85da-b18066d4ef01 None admin] PATH_INFO: `/v3/users` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:34.231618 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a90faf4-d03e-417d-85da-b18066d4ef01 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:47:34.231981 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a90faf4-d03e-417d-85da-b18066d4ef01 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:34.233453 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a90faf4-d03e-417d-85da-b18066d4ef01 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:34.242686 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:47:34.249076 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13e1e8b-493f-4530-ab11-4b07a8e0feb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:34.249716 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13e1e8b-493f-4530-ab11-4b07a8e0feb3 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:34.249992 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13e1e8b-493f-4530-ab11-4b07a8e0feb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:34.250257 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13e1e8b-493f-4530-ab11-4b07a8e0feb3 None admin] PATH_INFO: `/v3/services` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:34.250855 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13e1e8b-493f-4530-ab11-4b07a8e0feb3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:34.252209 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13e1e8b-493f-4530-ab11-4b07a8e0feb3 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:34.257773 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 519/1038] 10.222.0.20 () {56 vars in 1133 bytes} [Tue Aug 30 15:47:34 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:47:34.258834 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a90faf4-d03e-417d-85da-b18066d4ef01 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:34.259475 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 520/1039] 10.222.0.20 () {56 vars in 1087 bytes} [Tue Aug 30 15:47:34 2022] GET /identity/v3/services => generated 1957 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 15:47:34.274383 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:47:34.373951 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a90faf4-d03e-417d-85da-b18066d4ef01 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:34.412652 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc0de13-bcd1-44da-aca2-8ae83818b738 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:34.413293 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc0de13-bcd1-44da-aca2-8ae83818b738 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:34.413562 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc0de13-bcd1-44da-aca2-8ae83818b738 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:34.413833 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc0de13-bcd1-44da-aca2-8ae83818b738 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:34.482047 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc0de13-bcd1-44da-aca2-8ae83818b738 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:34.483897 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc0de13-bcd1-44da-aca2-8ae83818b738 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:34.493780 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:47:34.585311 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 520/1040] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Aug 30 15:47:34 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 30 15:47:34.591629 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13e1e8b-493f-4530-ab11-4b07a8e0feb3 None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:34.703835 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13e1e8b-493f-4530-ab11-4b07a8e0feb3 None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:34.739128 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3c3bdc-0dff-46a7-8ac8-e2d0fe6758ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:34.739686 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3c3bdc-0dff-46a7-8ac8-e2d0fe6758ab None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:34.739938 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3c3bdc-0dff-46a7-8ac8-e2d0fe6758ab None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:34.740209 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3c3bdc-0dff-46a7-8ac8-e2d0fe6758ab None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:34.751417 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3c3bdc-0dff-46a7-8ac8-e2d0fe6758ab None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:34.752863 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3c3bdc-0dff-46a7-8ac8-e2d0fe6758ab None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:34.760578 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4a7f2b-fdd9-431c-92e7-6a73ac59762f None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:34.760879 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4a7f2b-fdd9-431c-92e7-6a73ac59762f None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:34.761131 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4a7f2b-fdd9-431c-92e7-6a73ac59762f None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:34.762725 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 521/1041] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 30 15:47:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 15:47:34.767591 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112d4878-3b84-4328-b91c-aa0f63b5a9e5 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:34.767876 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112d4878-3b84-4328-b91c-aa0f63b5a9e5 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:34.767975 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application [None req-dc3c3bdc-0dff-46a7-8ac8-e2d0fe6758ab None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 15:47:34.767975 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:47:34.767975 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:47:34.767975 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:47:34.767975 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:47:34.767975 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:47:34.767975 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:47:34.767975 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:47:34.767975 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:47:34.767975 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:47:34.767975 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:47:34.767975 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:47:34.767975 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 15:47:34.767975 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 15:47:34.767975 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 15:47:34.767975 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 15:47:34.768882 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:47:34.768882 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:47:34.768882 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:47:34.768882 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:47:34.768882 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:47:34.768882 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:47:34.768882 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:47:34.768882 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application with Lock( Aug 30 15:47:34.768882 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:47:34.768882 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:47:34.768882 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:47:34.768882 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:47:34.768882 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:47:34.768882 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:47:34.768882 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:47:34.768882 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:47:34.768882 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 15:47:34.769825 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 15:47:34.769825 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 15:47:34.769825 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Aug 30 15:47:34.770016 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112d4878-3b84-4328-b91c-aa0f63b5a9e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:34.773582 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 521/1042] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 30 15:47:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 15:47:34.777564 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3c3bdc-0dff-46a7-8ac8-e2d0fe6758ab None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:34.787132 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-112d4878-3b84-4328-b91c-aa0f63b5a9e5 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:47:34.788341 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-112d4878-3b84-4328-b91c-aa0f63b5a9e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:47:34.832847 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:47:34.859042 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 522/1043] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 30 15:47:34 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 30 15:47:34.886135 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3c3bdc-0dff-46a7-8ac8-e2d0fe6758ab None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:34.921573 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa4819a-9ce2-4645-ae90-93af5c7960ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:34.922541 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa4819a-9ce2-4645-ae90-93af5c7960ac None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:34.923040 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa4819a-9ce2-4645-ae90-93af5c7960ac None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:34.923551 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa4819a-9ce2-4645-ae90-93af5c7960ac None admin] PATH_INFO: `/v3/domains` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:34.924384 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa4819a-9ce2-4645-ae90-93af5c7960ac None admin] RBAC: Adding query filter params (name=Default) {{(pid=81512) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:47:34.924878 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa4819a-9ce2-4645-ae90-93af5c7960ac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:34.926479 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa4819a-9ce2-4645-ae90-93af5c7960ac None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:34.939947 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 522/1044] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Aug 30 15:47:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 15:47:34.945719 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112d4878-3b84-4328-b91c-aa0f63b5a9e5 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:34.987295 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:47:35.047355 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112d4878-3b84-4328-b91c-aa0f63b5a9e5 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:35.080837 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcd5fd9-eff0-42ed-b3b2-dc05b1e69811 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:35.081762 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcd5fd9-eff0-42ed-b3b2-dc05b1e69811 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:35.082242 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcd5fd9-eff0-42ed-b3b2-dc05b1e69811 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:35.082705 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcd5fd9-eff0-42ed-b3b2-dc05b1e69811 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:35.089510 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcd5fd9-eff0-42ed-b3b2-dc05b1e69811 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:35.091202 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcd5fd9-eff0-42ed-b3b2-dc05b1e69811 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:35.102641 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-7fcd5fd9-eff0-42ed-b3b2-dc05b1e69811 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 15:47:35.102641 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:47:35.102641 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:47:35.102641 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:47:35.102641 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:47:35.102641 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:47:35.102641 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:47:35.102641 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:47:35.102641 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:47:35.102641 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:47:35.102641 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:47:35.102641 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:47:35.102641 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 15:47:35.102641 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 15:47:35.102641 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 15:47:35.103726 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 15:47:35.103726 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:47:35.103726 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:47:35.103726 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:47:35.103726 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:47:35.103726 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:47:35.103726 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:47:35.103726 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:47:35.103726 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Aug 30 15:47:35.103726 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:47:35.103726 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:47:35.103726 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:47:35.103726 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:47:35.103726 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:47:35.103726 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:47:35.103726 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:47:35.103726 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:47:35.104643 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 15:47:35.104643 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 15:47:35.104643 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 15:47:35.104643 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 15:47:35.104643 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 15:47:35.104643 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 15:47:35.104643 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 15:47:35.104643 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Aug 30 15:47:35.106568 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 523/1045] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Aug 30 15:47:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 15:47:35.112511 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa4819a-9ce2-4645-ae90-93af5c7960ac None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:35.127664 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcd5fd9-eff0-42ed-b3b2-dc05b1e69811 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:35.227998 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:47:35.232095 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcd5fd9-eff0-42ed-b3b2-dc05b1e69811 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:35.235248 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa4819a-9ce2-4645-ae90-93af5c7960ac None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:35.245158 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:47:35.268984 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c881fd-5db6-45dc-9fc2-35f1e00537a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:35.269663 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c881fd-5db6-45dc-9fc2-35f1e00537a1 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:35.269938 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c881fd-5db6-45dc-9fc2-35f1e00537a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:35.270211 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c881fd-5db6-45dc-9fc2-35f1e00537a1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:35.271573 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f06a84-ad03-493f-9f7f-a01faed3c4fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:35.272180 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f06a84-ad03-493f-9f7f-a01faed3c4fa None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:35.272454 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f06a84-ad03-493f-9f7f-a01faed3c4fa None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:35.272880 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f06a84-ad03-493f-9f7f-a01faed3c4fa None admin] PATH_INFO: `/v3/projects` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:35.273432 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f06a84-ad03-493f-9f7f-a01faed3c4fa None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81512) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:47:35.273736 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f06a84-ad03-493f-9f7f-a01faed3c4fa None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:35.275166 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f06a84-ad03-493f-9f7f-a01faed3c4fa None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:35.281094 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c881fd-5db6-45dc-9fc2-35f1e00537a1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 15:47:35.285903 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-54c881fd-5db6-45dc-9fc2-35f1e00537a1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 15:47:35.285903 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:47:35.285903 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:47:35.285903 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:47:35.285903 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:47:35.285903 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:47:35.285903 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:47:35.285903 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:47:35.285903 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:47:35.285903 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:47:35.285903 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:47:35.285903 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:47:35.285903 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 15:47:35.285903 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 15:47:35.285903 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 15:47:35.285903 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 15:47:35.286966 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 15:47:35.286966 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 15:47:35.286966 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 15:47:35.286966 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 15:47:35.286966 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:47:35.286966 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:47:35.286966 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:47:35.286966 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:47:35.286966 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:47:35.286966 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:47:35.286966 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:47:35.286966 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Aug 30 15:47:35.286966 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:47:35.286966 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:47:35.286966 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:47:35.286966 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:47:35.293138 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:47:35.293138 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:47:35.293138 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:47:35.293138 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:47:35.293138 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 15:47:35.293138 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 15:47:35.293138 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 15:47:35.293138 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 15:47:35.293138 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 15:47:35.293138 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 15:47:35.293138 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 15:47:35.293138 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Aug 30 15:47:35.293138 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 524/1046] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Aug 30 15:47:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 15:47:35.293891 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 523/1047] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Aug 30 15:47:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:47:35.293973 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c881fd-5db6-45dc-9fc2-35f1e00537a1 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:35.295320 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f06a84-ad03-493f-9f7f-a01faed3c4fa None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:35.395216 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c881fd-5db6-45dc-9fc2-35f1e00537a1 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:35.397687 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f06a84-ad03-493f-9f7f-a01faed3c4fa None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:35.429848 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46715532-d2e5-43be-9a85-683dce716c24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:35.430459 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46715532-d2e5-43be-9a85-683dce716c24 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:35.430738 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46715532-d2e5-43be-9a85-683dce716c24 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:35.431007 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46715532-d2e5-43be-9a85-683dce716c24 None admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:35.431668 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46715532-d2e5-43be-9a85-683dce716c24 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:47:35.431966 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46715532-d2e5-43be-9a85-683dce716c24 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:35.433437 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46715532-d2e5-43be-9a85-683dce716c24 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:35.434799 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235be64e-6381-491b-b5b7-2c55a236ef81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:35.435715 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235be64e-6381-491b-b5b7-2c55a236ef81 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:35.436056 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235be64e-6381-491b-b5b7-2c55a236ef81 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:35.436350 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235be64e-6381-491b-b5b7-2c55a236ef81 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:35.437156 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235be64e-6381-491b-b5b7-2c55a236ef81 None admin] RBAC: Adding query filter params (role.id=a89e0730512d449ca32479022aaf58d1, scope.project.id=854fe71b49224dff9a4b1766a20ab266, user.id=7a1e756ae9be4a7bac1e81acf07e8f3b) {{(pid=81512) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:47:35.437556 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235be64e-6381-491b-b5b7-2c55a236ef81 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:35.438978 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235be64e-6381-491b-b5b7-2c55a236ef81 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:35.442693 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 525/1048] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Aug 30 15:47:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:47:35.453252 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 524/1049] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Aug 30 15:47:35 2022] GET /identity/v3/role_assignments?user.id=7a1e756ae9be4a7bac1e81acf07e8f3b&scope.project.id=854fe71b49224dff9a4b1766a20ab266&role.id=a89e0730512d449ca32479022aaf58d1 => generated 259 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:47:35.453481 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46715532-d2e5-43be-9a85-683dce716c24 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:35.475033 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:47:35.560636 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46715532-d2e5-43be-9a85-683dce716c24 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:35.599858 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712b8514-82e2-4d0d-87cf-f12cc0f08285 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:35.600502 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712b8514-82e2-4d0d-87cf-f12cc0f08285 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:35.600775 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712b8514-82e2-4d0d-87cf-f12cc0f08285 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:35.601045 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712b8514-82e2-4d0d-87cf-f12cc0f08285 None admin] PATH_INFO: `/v3/users/c97a34d0464f4f728681f187e751df14` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:35.615333 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712b8514-82e2-4d0d-87cf-f12cc0f08285 None admin] RBAC: Authorizing `identity:get_user(user_id=c97a34d0464f4f728681f187e751df14)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:35.616791 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712b8514-82e2-4d0d-87cf-f12cc0f08285 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:35.619929 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 526/1050] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Aug 30 15:47:35 2022] GET /identity/v3/users/c97a34d0464f4f728681f187e751df14 => generated 285 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 15:47:35.625160 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235be64e-6381-491b-b5b7-2c55a236ef81 None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:35.667637 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:47:35.740050 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235be64e-6381-491b-b5b7-2c55a236ef81 None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:35.774057 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f094e065-9b72-4886-9bad-f1f33adc634d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:35.774668 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f094e065-9b72-4886-9bad-f1f33adc634d None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:35.774942 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f094e065-9b72-4886-9bad-f1f33adc634d None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:35.775205 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f094e065-9b72-4886-9bad-f1f33adc634d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:35.842393 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f094e065-9b72-4886-9bad-f1f33adc634d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:35.843841 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f094e065-9b72-4886-9bad-f1f33adc634d None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:35.874297 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:47:35.945113 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 525/1051] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Aug 30 15:47:35 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 30 15:47:35.951022 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712b8514-82e2-4d0d-87cf-f12cc0f08285 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:36.033586 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:47:36.056681 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712b8514-82e2-4d0d-87cf-f12cc0f08285 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:36.091792 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976fa2dd-26d3-4b80-8a20-5d4e843f3ba3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:36.092427 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976fa2dd-26d3-4b80-8a20-5d4e843f3ba3 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:36.092697 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976fa2dd-26d3-4b80-8a20-5d4e843f3ba3 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:36.092960 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976fa2dd-26d3-4b80-8a20-5d4e843f3ba3 None admin] PATH_INFO: `/v3/projects/1bf9d57f0be14002ba76d32c9b9db4d2` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:36.094139 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976fa2dd-26d3-4b80-8a20-5d4e843f3ba3 None admin] RBAC: Authorizing `identity:get_project(project_id=1bf9d57f0be14002ba76d32c9b9db4d2)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:36.095560 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976fa2dd-26d3-4b80-8a20-5d4e843f3ba3 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:36.102432 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 527/1052] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Aug 30 15:47:35 2022] GET /identity/v3/projects/1bf9d57f0be14002ba76d32c9b9db4d2 => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:47:36.103677 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f094e065-9b72-4886-9bad-f1f33adc634d None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:36.182514 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dbcd9e-5ac8-4b99-9057-8744d51b2909 None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:36.182851 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dbcd9e-5ac8-4b99-9057-8744d51b2909 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:36.183173 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dbcd9e-5ac8-4b99-9057-8744d51b2909 None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:36.184636 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 528/1053] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 30 15:47:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 15:47:36.189424 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8701ae-7e52-4ffd-ad0b-4aa07d2769d8 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:36.189718 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8701ae-7e52-4ffd-ad0b-4aa07d2769d8 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:36.189985 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8701ae-7e52-4ffd-ad0b-4aa07d2769d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:36.206455 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f094e065-9b72-4886-9bad-f1f33adc634d None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:36.208092 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-ed8701ae-7e52-4ffd-ad0b-4aa07d2769d8 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:47:36.209305 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-ed8701ae-7e52-4ffd-ad0b-4aa07d2769d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:47:36.236862 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:47:36.242568 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7250e795-d6b9-42d0-ba2f-0b03c4232c70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:36.243167 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7250e795-d6b9-42d0-ba2f-0b03c4232c70 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:36.243452 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7250e795-d6b9-42d0-ba2f-0b03c4232c70 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:36.243985 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7250e795-d6b9-42d0-ba2f-0b03c4232c70 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:36.245052 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7250e795-d6b9-42d0-ba2f-0b03c4232c70 None admin] RBAC: Adding query filter params (role.id=f265386950a641ac8413b53eda4ae34d, scope.project.id=1bf9d57f0be14002ba76d32c9b9db4d2, user.id=c97a34d0464f4f728681f187e751df14) {{(pid=81512) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:47:36.245401 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7250e795-d6b9-42d0-ba2f-0b03c4232c70 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:36.246838 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7250e795-d6b9-42d0-ba2f-0b03c4232c70 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:36.252386 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 526/1054] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Aug 30 15:47:36 2022] GET /identity/v3/role_assignments?user.id=c97a34d0464f4f728681f187e751df14&scope.project.id=1bf9d57f0be14002ba76d32c9b9db4d2&role.id=f265386950a641ac8413b53eda4ae34d => generated 259 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:47:36.282188 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 529/1055] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 30 15:47:36 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 30 15:47:36.546579 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7250e795-d6b9-42d0-ba2f-0b03c4232c70 None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:36.655927 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7250e795-d6b9-42d0-ba2f-0b03c4232c70 None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:36.691584 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43dba253-e485-4423-87f3-d38e37a9fc0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:36.692578 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dba253-e485-4423-87f3-d38e37a9fc0f None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:36.693074 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dba253-e485-4423-87f3-d38e37a9fc0f None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:36.693590 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dba253-e485-4423-87f3-d38e37a9fc0f None admin] PATH_INFO: `/v3/services/image` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:36.694371 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43dba253-e485-4423-87f3-d38e37a9fc0f None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:36.696011 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43dba253-e485-4423-87f3-d38e37a9fc0f None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:36.708856 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application [None req-43dba253-e485-4423-87f3-d38e37a9fc0f None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 30 15:47:36.708856 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:47:36.708856 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 15:47:36.708856 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 15:47:36.708856 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 15:47:36.708856 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 15:47:36.708856 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 15:47:36.708856 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 15:47:36.708856 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 30 15:47:36.708856 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Aug 30 15:47:36.708856 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 15:47:36.708856 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Aug 30 15:47:36.708856 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:47:36.708856 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:47:36.708856 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:47:36.708856 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:47:36.708856 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:47:36.709969 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:47:36.709969 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:47:36.709969 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:47:36.709969 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:47:36.709969 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:47:36.709969 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:47:36.709969 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 15:47:36.709969 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 15:47:36.709969 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 15:47:36.709969 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 15:47:36.709969 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:47:36.709969 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:47:36.709969 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:47:36.709969 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:47:36.709969 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:47:36.709969 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:47:36.710832 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:47:36.710832 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application with Lock( Aug 30 15:47:36.710832 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:47:36.710832 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:47:36.710832 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:47:36.710832 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:47:36.710832 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:47:36.710832 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:47:36.710832 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:47:36.710832 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:47:36.710832 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 15:47:36.710832 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 15:47:36.710832 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 30 15:47:36.710832 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Aug 30 15:47:36.712274 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 527/1056] 10.222.0.20 () {56 vars in 1105 bytes} [Tue Aug 30 15:47:36 2022] GET /identity/v3/services/image => generated 86 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 15:47:36.717266 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8701ae-7e52-4ffd-ad0b-4aa07d2769d8 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:36.797497 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:47:36.822742 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8701ae-7e52-4ffd-ad0b-4aa07d2769d8 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:36.859274 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97210bbf-642f-43cd-9caf-e553e027a266 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:36.859895 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97210bbf-642f-43cd-9caf-e553e027a266 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:36.860170 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97210bbf-642f-43cd-9caf-e553e027a266 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:36.860438 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97210bbf-642f-43cd-9caf-e553e027a266 None admin] PATH_INFO: `/v3/services` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:36.861069 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97210bbf-642f-43cd-9caf-e553e027a266 None admin] RBAC: Adding query filter params (name=image) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:47:36.861397 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97210bbf-642f-43cd-9caf-e553e027a266 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:36.862787 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97210bbf-642f-43cd-9caf-e553e027a266 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:36.868231 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 530/1057] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 30 15:47:36 2022] GET /identity/v3/services?name=image => generated 122 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:47:36.874285 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43dba253-e485-4423-87f3-d38e37a9fc0f None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:36.935728 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:47:36.978138 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43dba253-e485-4423-87f3-d38e37a9fc0f None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:37.013594 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2b4995-4cc5-4736-a117-56c0debbc114 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:37.014197 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2b4995-4cc5-4736-a117-56c0debbc114 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:37.014475 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2b4995-4cc5-4736-a117-56c0debbc114 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:37.014743 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2b4995-4cc5-4736-a117-56c0debbc114 None admin] PATH_INFO: `/v3/services` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:37.015369 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2b4995-4cc5-4736-a117-56c0debbc114 None admin] RBAC: Adding query filter params (type=image) {{(pid=81512) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:47:37.015671 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2b4995-4cc5-4736-a117-56c0debbc114 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:37.017048 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2b4995-4cc5-4736-a117-56c0debbc114 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:37.022154 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 528/1058] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 30 15:47:36 2022] GET /identity/v3/services?type=image => generated 350 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:47:37.027753 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97210bbf-642f-43cd-9caf-e553e027a266 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:37.132351 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97210bbf-642f-43cd-9caf-e553e027a266 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:37.165823 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4fd24a-de28-45ff-92d5-aa1fa3ca4b00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:37.166428 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4fd24a-de28-45ff-92d5-aa1fa3ca4b00 None admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:37.166697 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4fd24a-de28-45ff-92d5-aa1fa3ca4b00 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:37.166959 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4fd24a-de28-45ff-92d5-aa1fa3ca4b00 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:37.167624 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4fd24a-de28-45ff-92d5-aa1fa3ca4b00 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:37.168986 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4fd24a-de28-45ff-92d5-aa1fa3ca4b00 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:37.179803 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:47:37.194767 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 531/1059] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Aug 30 15:47:37 2022] POST /identity/v3/endpoints => generated 327 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 15:47:37.339652 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e356f913-7b2a-46cf-81e1-266623876d44 None None] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:37.340536 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e356f913-7b2a-46cf-81e1-266623876d44 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:37.340777 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e356f913-7b2a-46cf-81e1-266623876d44 None None] PATH_INFO: `/` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:37.342479 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 529/1060] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 30 15:47:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 15:47:37.347386 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf41ec1e-7a5d-421f-a331-70642c26b4b3 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:37.347804 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf41ec1e-7a5d-421f-a331-70642c26b4b3 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:37.347996 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf41ec1e-7a5d-421f-a331-70642c26b4b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:37.367147 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-bf41ec1e-7a5d-421f-a331-70642c26b4b3 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:47:37.368326 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-bf41ec1e-7a5d-421f-a331-70642c26b4b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:47:37.444831 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 532/1061] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 30 15:47:37 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 15:47:37.707992 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e356f913-7b2a-46cf-81e1-266623876d44 None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:37.810627 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e356f913-7b2a-46cf-81e1-266623876d44 None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:37.834467 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:47:37.847248 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefb8a23-aab4-4226-b700-380c4749b20c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:37.847884 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefb8a23-aab4-4226-b700-380c4749b20c None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:37.848161 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefb8a23-aab4-4226-b700-380c4749b20c None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:37.848429 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefb8a23-aab4-4226-b700-380c4749b20c None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:37.858903 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fefb8a23-aab4-4226-b700-380c4749b20c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 15:47:37.863689 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application [None req-fefb8a23-aab4-4226-b700-380c4749b20c None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 30 15:47:37.863689 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:47:37.863689 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:47:37.863689 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:47:37.863689 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:47:37.863689 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:47:37.863689 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:47:37.863689 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:47:37.863689 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:47:37.863689 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:47:37.863689 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:47:37.863689 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:47:37.863689 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 15:47:37.863689 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 15:47:37.863689 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 15:47:37.863689 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 15:47:37.864614 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 15:47:37.864614 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 15:47:37.864614 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 15:47:37.864614 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 15:47:37.864614 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:47:37.864614 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:47:37.864614 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:47:37.864614 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:47:37.864614 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:47:37.864614 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:47:37.864614 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:47:37.864614 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application with Lock( Aug 30 15:47:37.864614 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:47:37.864614 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:47:37.864614 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:47:37.864614 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:47:37.866056 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:47:37.866056 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:47:37.866056 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:47:37.866056 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:47:37.866056 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 15:47:37.866056 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 15:47:37.866056 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 15:47:37.866056 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 15:47:37.866056 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 15:47:37.866056 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 15:47:37.866056 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 30 15:47:37.866056 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Aug 30 15:47:37.866056 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 530/1062] 10.222.0.20 () {56 vars in 1120 bytes} [Tue Aug 30 15:47:37 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 15:47:37.871192 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf41ec1e-7a5d-421f-a331-70642c26b4b3 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:37.971163 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf41ec1e-7a5d-421f-a331-70642c26b4b3 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:37.994366 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:47:38.008490 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309ff50a-f329-4cac-a0fa-741eae094651 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:38.009110 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309ff50a-f329-4cac-a0fa-741eae094651 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:38.009415 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309ff50a-f329-4cac-a0fa-741eae094651 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:38.009684 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309ff50a-f329-4cac-a0fa-741eae094651 None admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:38.010354 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309ff50a-f329-4cac-a0fa-741eae094651 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:47:38.010657 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309ff50a-f329-4cac-a0fa-741eae094651 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:38.012027 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309ff50a-f329-4cac-a0fa-741eae094651 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:38.020531 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 533/1063] 10.222.0.20 () {56 vars in 1115 bytes} [Tue Aug 30 15:47:37 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:47:38.026191 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefb8a23-aab4-4226-b700-380c4749b20c None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:38.140247 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefb8a23-aab4-4226-b700-380c4749b20c None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:38.151583 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4eb22af-c4cd-4d5c-8369-5fea68effde0 None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:38.151928 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4eb22af-c4cd-4d5c-8369-5fea68effde0 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:38.152178 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4eb22af-c4cd-4d5c-8369-5fea68effde0 None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:38.157281 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 534/1064] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 30 15:47:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 15:47:38.158952 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f912c44-14f5-47e9-a4ef-bd84f26607ae None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:38.159234 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f912c44-14f5-47e9-a4ef-bd84f26607ae None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:38.159713 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f912c44-14f5-47e9-a4ef-bd84f26607ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:38.180060 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-0f912c44-14f5-47e9-a4ef-bd84f26607ae None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:47:38.181406 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-0f912c44-14f5-47e9-a4ef-bd84f26607ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:47:38.189013 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4631b0a-e1f0-459a-a035-850729efa9ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:38.190104 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4631b0a-e1f0-459a-a035-850729efa9ed None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:38.190462 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4631b0a-e1f0-459a-a035-850729efa9ed None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:38.190651 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4631b0a-e1f0-459a-a035-850729efa9ed None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:38.201690 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4631b0a-e1f0-459a-a035-850729efa9ed None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:38.203158 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4631b0a-e1f0-459a-a035-850729efa9ed None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:38.218608 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application [None req-d4631b0a-e1f0-459a-a035-850729efa9ed None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 15:47:38.218608 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:47:38.218608 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:47:38.218608 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:47:38.218608 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:47:38.218608 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:47:38.218608 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:47:38.218608 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:47:38.218608 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:47:38.218608 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:47:38.218608 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:47:38.218608 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:47:38.218608 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 15:47:38.218608 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 15:47:38.218608 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 15:47:38.218608 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 15:47:38.219663 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:47:38.219663 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:47:38.219663 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:47:38.219663 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:47:38.219663 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:47:38.219663 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:47:38.219663 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:47:38.219663 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application with Lock( Aug 30 15:47:38.219663 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:47:38.219663 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:47:38.219663 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:47:38.219663 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:47:38.219663 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:47:38.219663 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:47:38.219663 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:47:38.219663 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:47:38.219663 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 15:47:38.221274 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 15:47:38.221274 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 15:47:38.221274 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Aug 30 15:47:38.221274 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 531/1065] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 30 15:47:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 15:47:38.225544 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4631b0a-e1f0-459a-a035-850729efa9ed None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:38.257658 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 535/1066] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 30 15:47:38 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 15:47:38.264497 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:47:38.333400 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4631b0a-e1f0-459a-a035-850729efa9ed None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:38.368277 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8cecdf-79f0-4678-93a5-7fbf2c07bf01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:38.368935 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8cecdf-79f0-4678-93a5-7fbf2c07bf01 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:38.369145 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8cecdf-79f0-4678-93a5-7fbf2c07bf01 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:38.369751 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8cecdf-79f0-4678-93a5-7fbf2c07bf01 None admin] PATH_INFO: `/v3/domains` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:38.370415 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8cecdf-79f0-4678-93a5-7fbf2c07bf01 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81512) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:47:38.370717 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8cecdf-79f0-4678-93a5-7fbf2c07bf01 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:38.372087 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8cecdf-79f0-4678-93a5-7fbf2c07bf01 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:38.383798 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 532/1067] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Aug 30 15:47:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 15:47:38.389473 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f912c44-14f5-47e9-a4ef-bd84f26607ae None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:38.487778 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f912c44-14f5-47e9-a4ef-bd84f26607ae None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:38.503040 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:47:38.520291 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8cecdf-79f0-4678-93a5-7fbf2c07bf01 None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:38.525208 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c492222c-bc3f-4572-8b5f-7888dc7f9676 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:38.525874 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c492222c-bc3f-4572-8b5f-7888dc7f9676 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:38.526152 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c492222c-bc3f-4572-8b5f-7888dc7f9676 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:38.526419 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c492222c-bc3f-4572-8b5f-7888dc7f9676 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:38.533938 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c492222c-bc3f-4572-8b5f-7888dc7f9676 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:38.535353 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c492222c-bc3f-4572-8b5f-7888dc7f9676 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:38.547349 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-c492222c-bc3f-4572-8b5f-7888dc7f9676 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 30 15:47:38.547349 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:47:38.547349 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:47:38.547349 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:47:38.547349 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:47:38.547349 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:47:38.547349 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:47:38.547349 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:47:38.547349 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:47:38.547349 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:47:38.547349 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:47:38.547349 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:47:38.547349 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 15:47:38.547349 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 15:47:38.547349 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 15:47:38.547349 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 15:47:38.548507 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:47:38.548507 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:47:38.548507 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 15:47:38.548507 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 15:47:38.548507 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 15:47:38.548507 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 15:47:38.548507 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:47:38.548507 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:47:38.548507 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:47:38.548507 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:47:38.548507 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:47:38.548507 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Aug 30 15:47:38.548507 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:47:38.548507 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:47:38.548507 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:47:38.548507 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:47:38.548507 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:47:38.548507 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:47:38.550239 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:47:38.550239 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:47:38.550239 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 15:47:38.550239 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 15:47:38.550239 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 15:47:38.550239 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 15:47:38.550239 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 15:47:38.550239 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 15:47:38.550239 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 30 15:47:38.550239 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Aug 30 15:47:38.550239 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 536/1068] 10.222.0.20 () {56 vars in 1093 bytes} [Tue Aug 30 15:47:38 2022] GET /identity/v3/users/nova => generated 82 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 15:47:38.554066 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c492222c-bc3f-4572-8b5f-7888dc7f9676 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:38.634531 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:47:38.641822 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8cecdf-79f0-4678-93a5-7fbf2c07bf01 None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:38.665152 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c492222c-bc3f-4572-8b5f-7888dc7f9676 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:38.677896 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d821e2a1-4600-4fda-9b8f-fda2ae54869d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:38.678532 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d821e2a1-4600-4fda-9b8f-fda2ae54869d None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:38.678810 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d821e2a1-4600-4fda-9b8f-fda2ae54869d None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:38.679071 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d821e2a1-4600-4fda-9b8f-fda2ae54869d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:38.691220 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d821e2a1-4600-4fda-9b8f-fda2ae54869d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 15:47:38.695895 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application [None req-d821e2a1-4600-4fda-9b8f-fda2ae54869d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 15:47:38.695895 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:47:38.695895 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:47:38.695895 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:47:38.695895 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:47:38.695895 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:47:38.695895 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:47:38.695895 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:47:38.695895 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:47:38.695895 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:47:38.695895 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:47:38.695895 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:47:38.695895 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 15:47:38.695895 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 15:47:38.695895 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 15:47:38.695895 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 15:47:38.696838 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 15:47:38.696838 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 15:47:38.696838 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 15:47:38.696838 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 15:47:38.696838 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:47:38.696838 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:47:38.696838 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:47:38.696838 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:47:38.696838 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:47:38.696838 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:47:38.696838 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:47:38.696838 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application with Lock( Aug 30 15:47:38.696838 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:47:38.696838 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:47:38.696838 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:47:38.696838 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:47:38.697798 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:47:38.697798 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:47:38.697798 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:47:38.697798 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:47:38.697798 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 15:47:38.697798 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 15:47:38.697798 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 15:47:38.697798 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 15:47:38.697798 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 15:47:38.697798 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 15:47:38.697798 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 15:47:38.697798 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Aug 30 15:47:38.702136 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 533/1069] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Aug 30 15:47:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 15:47:38.703209 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d821e2a1-4600-4fda-9b8f-fda2ae54869d None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:38.703724 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2a43a9-b72e-4eb8-82c6-76d02f46dd8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:38.704348 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2a43a9-b72e-4eb8-82c6-76d02f46dd8b None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:38.704628 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2a43a9-b72e-4eb8-82c6-76d02f46dd8b None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:38.704899 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2a43a9-b72e-4eb8-82c6-76d02f46dd8b None admin] PATH_INFO: `/v3/users` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:38.705669 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2a43a9-b72e-4eb8-82c6-76d02f46dd8b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:47:38.706151 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2a43a9-b72e-4eb8-82c6-76d02f46dd8b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:38.708491 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2a43a9-b72e-4eb8-82c6-76d02f46dd8b None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:38.735760 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 537/1070] 10.222.0.20 () {56 vars in 1133 bytes} [Tue Aug 30 15:47:38 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 15:47:38.736856 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2a43a9-b72e-4eb8-82c6-76d02f46dd8b None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:38.775576 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:47:38.808023 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d821e2a1-4600-4fda-9b8f-fda2ae54869d None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:38.848371 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f44b086-8028-4d36-a01c-993ba398153b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:38.849081 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f44b086-8028-4d36-a01c-993ba398153b None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:38.849384 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f44b086-8028-4d36-a01c-993ba398153b None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:38.849741 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f44b086-8028-4d36-a01c-993ba398153b None admin] PATH_INFO: `/v3/roles` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:38.850410 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f44b086-8028-4d36-a01c-993ba398153b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81512) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:47:38.850695 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f44b086-8028-4d36-a01c-993ba398153b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:38.852074 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f44b086-8028-4d36-a01c-993ba398153b None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:38.852769 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2a43a9-b72e-4eb8-82c6-76d02f46dd8b None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:38.865563 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 534/1071] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Aug 30 15:47:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:47:38.867031 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f44b086-8028-4d36-a01c-993ba398153b None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:38.886526 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:47:38.891018 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48dd39e9-444a-4a46-af1c-6179a94fc29d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:38.891664 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dd39e9-444a-4a46-af1c-6179a94fc29d None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:38.891942 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dd39e9-444a-4a46-af1c-6179a94fc29d None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:38.892214 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dd39e9-444a-4a46-af1c-6179a94fc29d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:38.902389 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48dd39e9-444a-4a46-af1c-6179a94fc29d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:38.903927 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48dd39e9-444a-4a46-af1c-6179a94fc29d None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:38.918948 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-48dd39e9-444a-4a46-af1c-6179a94fc29d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 15:47:38.918948 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:47:38.918948 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:47:38.918948 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:47:38.918948 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:47:38.918948 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:47:38.918948 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:47:38.918948 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:47:38.918948 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:47:38.918948 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:47:38.918948 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:47:38.918948 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:47:38.918948 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 15:47:38.918948 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 15:47:38.918948 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 15:47:38.918948 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 15:47:38.919984 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:47:38.919984 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:47:38.919984 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:47:38.919984 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:47:38.919984 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:47:38.919984 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:47:38.919984 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:47:38.919984 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Aug 30 15:47:38.919984 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:47:38.919984 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:47:38.919984 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:47:38.919984 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:47:38.919984 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:47:38.919984 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:47:38.919984 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:47:38.919984 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:47:38.919984 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 15:47:38.921009 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 15:47:38.921009 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 15:47:38.921009 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Aug 30 15:47:38.921009 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 538/1072] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 30 15:47:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 15:47:38.925976 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48dd39e9-444a-4a46-af1c-6179a94fc29d None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:38.977890 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f44b086-8028-4d36-a01c-993ba398153b None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:39.012471 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bedceda-8ef8-4ea8-804d-747bdfa18abe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:39.013134 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bedceda-8ef8-4ea8-804d-747bdfa18abe None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:39.013445 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bedceda-8ef8-4ea8-804d-747bdfa18abe None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:39.013718 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bedceda-8ef8-4ea8-804d-747bdfa18abe None admin] PATH_INFO: `/v3/users/c97a34d0464f4f728681f187e751df14` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:39.014724 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:47:39.014843 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bedceda-8ef8-4ea8-804d-747bdfa18abe None admin] RBAC: Authorizing `identity:get_user(user_id=c97a34d0464f4f728681f187e751df14)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:39.016294 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bedceda-8ef8-4ea8-804d-747bdfa18abe None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:39.023029 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 535/1073] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Aug 30 15:47:38 2022] GET /identity/v3/users/c97a34d0464f4f728681f187e751df14 => generated 285 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:47:39.023868 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bedceda-8ef8-4ea8-804d-747bdfa18abe None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:39.036710 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48dd39e9-444a-4a46-af1c-6179a94fc29d None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:39.071253 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448b5c90-27d3-4ac3-8bdc-7c67aa42cf21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:39.071875 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448b5c90-27d3-4ac3-8bdc-7c67aa42cf21 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:39.072153 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448b5c90-27d3-4ac3-8bdc-7c67aa42cf21 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:39.072428 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448b5c90-27d3-4ac3-8bdc-7c67aa42cf21 None admin] PATH_INFO: `/v3/domains` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:39.073074 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448b5c90-27d3-4ac3-8bdc-7c67aa42cf21 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:47:39.073732 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448b5c90-27d3-4ac3-8bdc-7c67aa42cf21 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:39.074776 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448b5c90-27d3-4ac3-8bdc-7c67aa42cf21 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:39.087047 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 539/1074] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Aug 30 15:47:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 15:47:39.092636 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448b5c90-27d3-4ac3-8bdc-7c67aa42cf21 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:39.115265 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:47:39.131852 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bedceda-8ef8-4ea8-804d-747bdfa18abe None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:39.167654 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14241d1-5267-4b00-9f88-f5c547498d45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:39.168647 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14241d1-5267-4b00-9f88-f5c547498d45 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:39.168927 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14241d1-5267-4b00-9f88-f5c547498d45 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:39.169522 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14241d1-5267-4b00-9f88-f5c547498d45 None admin] PATH_INFO: `/v3/projects/1bf9d57f0be14002ba76d32c9b9db4d2` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:39.170407 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14241d1-5267-4b00-9f88-f5c547498d45 None admin] RBAC: Authorizing `identity:get_project(project_id=1bf9d57f0be14002ba76d32c9b9db4d2)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:39.171853 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14241d1-5267-4b00-9f88-f5c547498d45 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:39.173553 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 536/1075] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Aug 30 15:47:39 2022] GET /identity/v3/projects/1bf9d57f0be14002ba76d32c9b9db4d2 => generated 311 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:47:39.174670 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537411d2-0ab5-4f18-a914-e438a307be4d None None] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:39.175017 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537411d2-0ab5-4f18-a914-e438a307be4d None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:39.175293 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537411d2-0ab5-4f18-a914-e438a307be4d None None] PATH_INFO: `/` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:39.176488 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 537/1076] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 30 15:47:39 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 15:47:39.179253 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537411d2-0ab5-4f18-a914-e438a307be4d None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:39.196352 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448b5c90-27d3-4ac3-8bdc-7c67aa42cf21 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:39.229378 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cbbfff-2d9d-4b6a-a041-12a088086081 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:39.229996 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cbbfff-2d9d-4b6a-a041-12a088086081 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:39.230277 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cbbfff-2d9d-4b6a-a041-12a088086081 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:39.230544 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cbbfff-2d9d-4b6a-a041-12a088086081 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:39.237185 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0cbbfff-2d9d-4b6a-a041-12a088086081 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:39.238674 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0cbbfff-2d9d-4b6a-a041-12a088086081 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:39.250414 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-d0cbbfff-2d9d-4b6a-a041-12a088086081 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 15:47:39.250414 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:47:39.250414 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:47:39.250414 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:47:39.250414 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:47:39.250414 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:47:39.250414 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:47:39.250414 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:47:39.250414 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:47:39.250414 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:47:39.250414 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:47:39.250414 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:47:39.250414 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 15:47:39.250414 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 15:47:39.250414 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 15:47:39.251342 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 15:47:39.251342 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:47:39.251342 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:47:39.251342 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:47:39.251342 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:47:39.251342 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:47:39.251342 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:47:39.251342 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:47:39.251342 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Aug 30 15:47:39.251342 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:47:39.251342 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:47:39.251342 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:47:39.251342 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:47:39.251342 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:47:39.251342 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:47:39.251342 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:47:39.251342 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:47:39.252255 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 15:47:39.252255 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 15:47:39.252255 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 15:47:39.252255 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 15:47:39.252255 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 15:47:39.252255 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 15:47:39.252255 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 15:47:39.252255 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Aug 30 15:47:39.252255 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 540/1077] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Aug 30 15:47:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 15:47:39.252751 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051747d5-2331-4a93-9c36-d63fb392e739 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:39.253262 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051747d5-2331-4a93-9c36-d63fb392e739 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:39.253262 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051747d5-2331-4a93-9c36-d63fb392e739 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:39.261203 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:47:39.272772 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-051747d5-2331-4a93-9c36-d63fb392e739 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:47:39.274028 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-051747d5-2331-4a93-9c36-d63fb392e739 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:47:39.282011 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537411d2-0ab5-4f18-a914-e438a307be4d None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:39.316561 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cb9acc-4037-4758-8145-8b76adb5adf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:39.317128 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cb9acc-4037-4758-8145-8b76adb5adf2 None admin] REQUEST_METHOD: `PUT` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:39.317443 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cb9acc-4037-4758-8145-8b76adb5adf2 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:39.317713 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cb9acc-4037-4758-8145-8b76adb5adf2 None admin] PATH_INFO: `/v3/projects/1bf9d57f0be14002ba76d32c9b9db4d2/users/c97a34d0464f4f728681f187e751df14/roles/f265386950a641ac8413b53eda4ae34d` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:39.319584 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0cb9acc-4037-4758-8145-8b76adb5adf2 None admin] RBAC: Authorizing `identity:create_grant(project_id=1bf9d57f0be14002ba76d32c9b9db4d2, user_id=c97a34d0464f4f728681f187e751df14, role_id=f265386950a641ac8413b53eda4ae34d)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:39.321014 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0cb9acc-4037-4758-8145-8b76adb5adf2 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:39.335112 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 538/1078] 10.222.0.20 () {58 vars in 1439 bytes} [Tue Aug 30 15:47:39 2022] PUT /identity/v3/projects/1bf9d57f0be14002ba76d32c9b9db4d2/users/c97a34d0464f4f728681f187e751df14/roles/f265386950a641ac8413b53eda4ae34d => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 15:47:39.335888 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cb9acc-4037-4758-8145-8b76adb5adf2 None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:39.357263 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 541/1079] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 30 15:47:39 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 15:47:39.357511 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:47:39.454711 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cb9acc-4037-4758-8145-8b76adb5adf2 None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:39.496494 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5052f509-006e-448c-b0dd-30746c8f32e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:39.497475 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5052f509-006e-448c-b0dd-30746c8f32e1 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:39.497924 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5052f509-006e-448c-b0dd-30746c8f32e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:39.498343 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5052f509-006e-448c-b0dd-30746c8f32e1 None admin] PATH_INFO: `/v3/projects` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:39.499052 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5052f509-006e-448c-b0dd-30746c8f32e1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81512) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:47:39.499359 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5052f509-006e-448c-b0dd-30746c8f32e1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:39.500797 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5052f509-006e-448c-b0dd-30746c8f32e1 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:39.512975 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 539/1080] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Aug 30 15:47:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:47:39.519211 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051747d5-2331-4a93-9c36-d63fb392e739 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:39.614804 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5052f509-006e-448c-b0dd-30746c8f32e1 None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:39.624712 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051747d5-2331-4a93-9c36-d63fb392e739 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:39.667930 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe6400b-e789-4137-8c4b-96fca0236be1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:39.668626 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe6400b-e789-4137-8c4b-96fca0236be1 None admin] REQUEST_METHOD: `PUT` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:39.668885 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe6400b-e789-4137-8c4b-96fca0236be1 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:39.669156 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe6400b-e789-4137-8c4b-96fca0236be1 None admin] PATH_INFO: `/v3/projects/854fe71b49224dff9a4b1766a20ab266/users/7a1e756ae9be4a7bac1e81acf07e8f3b/roles/a89e0730512d449ca32479022aaf58d1` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:39.671445 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe6400b-e789-4137-8c4b-96fca0236be1 None admin] RBAC: Authorizing `identity:create_grant(project_id=854fe71b49224dff9a4b1766a20ab266, user_id=7a1e756ae9be4a7bac1e81acf07e8f3b, role_id=a89e0730512d449ca32479022aaf58d1)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:39.672949 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe6400b-e789-4137-8c4b-96fca0236be1 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:39.675389 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:47:39.689983 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 542/1081] 10.222.0.20 () {58 vars in 1439 bytes} [Tue Aug 30 15:47:39 2022] PUT /identity/v3/projects/854fe71b49224dff9a4b1766a20ab266/users/7a1e756ae9be4a7bac1e81acf07e8f3b/roles/a89e0730512d449ca32479022aaf58d1 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 15:47:39.701363 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:47:39.730439 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5052f509-006e-448c-b0dd-30746c8f32e1 None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:39.769492 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7655bc54-6c94-478f-9098-e870070cbaf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:39.770130 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7655bc54-6c94-478f-9098-e870070cbaf9 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:39.770402 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7655bc54-6c94-478f-9098-e870070cbaf9 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:39.770659 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7655bc54-6c94-478f-9098-e870070cbaf9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:39.847549 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7655bc54-6c94-478f-9098-e870070cbaf9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:39.849053 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7655bc54-6c94-478f-9098-e870070cbaf9 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:39.928505 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:47:39.955507 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 540/1082] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Aug 30 15:47:39 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 30 15:47:39.961337 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe6400b-e789-4137-8c4b-96fca0236be1 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:40.062378 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe6400b-e789-4137-8c4b-96fca0236be1 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:40.095707 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6a483e-b75e-4ed2-8020-cd3c9c28ce4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:40.096333 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6a483e-b75e-4ed2-8020-cd3c9c28ce4b None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:40.096744 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6a483e-b75e-4ed2-8020-cd3c9c28ce4b None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:40.096982 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6a483e-b75e-4ed2-8020-cd3c9c28ce4b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:40.107241 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6a483e-b75e-4ed2-8020-cd3c9c28ce4b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:40.108686 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6a483e-b75e-4ed2-8020-cd3c9c28ce4b None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:40.123354 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-ce6a483e-b75e-4ed2-8020-cd3c9c28ce4b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 15:47:40.123354 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:47:40.123354 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:47:40.123354 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:47:40.123354 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:47:40.123354 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:47:40.123354 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:47:40.123354 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:47:40.123354 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:47:40.123354 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:47:40.123354 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:47:40.123354 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:47:40.123354 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 15:47:40.123354 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 15:47:40.123354 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 15:47:40.123354 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 15:47:40.124333 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:47:40.124333 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:47:40.124333 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:47:40.124333 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:47:40.124333 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:47:40.124333 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:47:40.124333 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:47:40.124333 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Aug 30 15:47:40.124333 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:47:40.124333 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:47:40.124333 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:47:40.124333 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:47:40.124333 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:47:40.124333 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:47:40.124333 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:47:40.124333 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:47:40.124333 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 15:47:40.125765 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 15:47:40.125765 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 15:47:40.125765 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Aug 30 15:47:40.129508 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 543/1083] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 30 15:47:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 15:47:40.130521 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7655bc54-6c94-478f-9098-e870070cbaf9 None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:40.234632 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7655bc54-6c94-478f-9098-e870070cbaf9 None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:40.269511 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91e23b8-ed8a-446c-8c19-75e398fd48da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:40.270112 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91e23b8-ed8a-446c-8c19-75e398fd48da None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:40.270386 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91e23b8-ed8a-446c-8c19-75e398fd48da None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:40.270647 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91e23b8-ed8a-446c-8c19-75e398fd48da None admin] PATH_INFO: `/v3/domains` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:40.271278 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91e23b8-ed8a-446c-8c19-75e398fd48da None admin] RBAC: Adding query filter params (name=Default) {{(pid=81512) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:47:40.271583 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91e23b8-ed8a-446c-8c19-75e398fd48da None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:40.272928 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91e23b8-ed8a-446c-8c19-75e398fd48da None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:40.284619 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 541/1084] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Aug 30 15:47:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:47:41.224373 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6ead1f-29f2-47e1-8e7b-84d3ac5f419b None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:41.225010 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6ead1f-29f2-47e1-8e7b-84d3ac5f419b None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:41.225010 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6ead1f-29f2-47e1-8e7b-84d3ac5f419b None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:41.226250 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 544/1085] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 30 15:47:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 15:47:41.231681 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496b1678-bafd-4fa4-83be-dc3d07800390 None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:41.232230 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496b1678-bafd-4fa4-83be-dc3d07800390 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:41.232230 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496b1678-bafd-4fa4-83be-dc3d07800390 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:41.251229 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-496b1678-bafd-4fa4-83be-dc3d07800390 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:47:41.252390 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-496b1678-bafd-4fa4-83be-dc3d07800390 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:47:41.279167 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:47:41.321478 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 542/1086] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 30 15:47:41 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 15:47:41.585963 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6ead1f-29f2-47e1-8e7b-84d3ac5f419b None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:41.597734 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2572616-459d-4ed4-8f48-16cd3fd244e8 None None] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:41.598010 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2572616-459d-4ed4-8f48-16cd3fd244e8 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:41.598301 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2572616-459d-4ed4-8f48-16cd3fd244e8 None None] PATH_INFO: `/` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:41.599605 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 543/1087] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 30 15:47:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 15:47:41.604675 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ccde2e-24c4-456e-b3de-e161d9e3b625 None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:41.604950 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ccde2e-24c4-456e-b3de-e161d9e3b625 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:41.605244 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ccde2e-24c4-456e-b3de-e161d9e3b625 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:41.623960 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-74ccde2e-24c4-456e-b3de-e161d9e3b625 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:47:41.625154 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-74ccde2e-24c4-456e-b3de-e161d9e3b625 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:47:41.637538 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:47:41.693089 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6ead1f-29f2-47e1-8e7b-84d3ac5f419b None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:41.699091 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 544/1088] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 30 15:47:41 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 30 15:47:41.725746 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8b58e9-531b-44bc-8e8a-0dc48e677d15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:41.726368 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8b58e9-531b-44bc-8e8a-0dc48e677d15 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:41.726642 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8b58e9-531b-44bc-8e8a-0dc48e677d15 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:41.726907 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8b58e9-531b-44bc-8e8a-0dc48e677d15 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:41.737444 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8b58e9-531b-44bc-8e8a-0dc48e677d15 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 15:47:41.742204 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-ac8b58e9-531b-44bc-8e8a-0dc48e677d15 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 15:47:41.742204 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:47:41.742204 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:47:41.742204 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:47:41.742204 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:47:41.742204 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:47:41.742204 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:47:41.742204 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:47:41.742204 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:47:41.742204 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:47:41.742204 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:47:41.742204 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:47:41.742204 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 15:47:41.742204 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 15:47:41.742204 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 15:47:41.742204 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 15:47:41.743314 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 15:47:41.743314 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 15:47:41.743314 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 15:47:41.743314 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 15:47:41.743314 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:47:41.743314 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:47:41.743314 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:47:41.743314 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:47:41.743314 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:47:41.743314 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:47:41.743314 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:47:41.743314 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Aug 30 15:47:41.743314 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:47:41.743314 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:47:41.743314 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:47:41.743314 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:47:41.744175 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:47:41.744175 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:47:41.744175 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:47:41.744175 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:47:41.744175 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 15:47:41.744175 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 15:47:41.744175 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 15:47:41.744175 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 15:47:41.744175 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 15:47:41.744175 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 15:47:41.744175 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 15:47:41.744175 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Aug 30 15:47:41.744175 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 545/1089] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Aug 30 15:47:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 15:47:41.749847 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ccde2e-24c4-456e-b3de-e161d9e3b625 None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:41.843961 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:47:41.851603 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ccde2e-24c4-456e-b3de-e161d9e3b625 None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:41.886588 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc39086-9969-4e54-a6f8-336bcfb19b3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:41.887198 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc39086-9969-4e54-a6f8-336bcfb19b3f None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:41.887475 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc39086-9969-4e54-a6f8-336bcfb19b3f None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:41.887744 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc39086-9969-4e54-a6f8-336bcfb19b3f None admin] PATH_INFO: `/v3/roles` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:41.888460 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc39086-9969-4e54-a6f8-336bcfb19b3f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81512) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:47:41.888765 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc39086-9969-4e54-a6f8-336bcfb19b3f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:41.890204 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc39086-9969-4e54-a6f8-336bcfb19b3f None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:41.898639 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 545/1090] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Aug 30 15:47:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:47:41.904744 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8b58e9-531b-44bc-8e8a-0dc48e677d15 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:41.959364 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc39086-9969-4e54-a6f8-336bcfb19b3f None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:42.011061 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8b58e9-531b-44bc-8e8a-0dc48e677d15 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:42.017854 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:47:42.045156 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a02a26c-57c8-4f76-829b-8b83b986f76e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:42.045806 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a02a26c-57c8-4f76-829b-8b83b986f76e None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:42.046100 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a02a26c-57c8-4f76-829b-8b83b986f76e None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:42.046324 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a02a26c-57c8-4f76-829b-8b83b986f76e None admin] PATH_INFO: `/v3/users/c97a34d0464f4f728681f187e751df14` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:42.047442 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a02a26c-57c8-4f76-829b-8b83b986f76e None admin] RBAC: Authorizing `identity:get_user(user_id=c97a34d0464f4f728681f187e751df14)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:42.048866 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a02a26c-57c8-4f76-829b-8b83b986f76e None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:42.050681 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 546/1091] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Aug 30 15:47:41 2022] GET /identity/v3/users/c97a34d0464f4f728681f187e751df14 => generated 285 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 15:47:42.056637 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a02a26c-57c8-4f76-829b-8b83b986f76e None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:42.062663 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc39086-9969-4e54-a6f8-336bcfb19b3f None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:42.095248 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa309cd-0a1d-40cd-8d18-c25d394d2728 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:42.095798 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa309cd-0a1d-40cd-8d18-c25d394d2728 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:42.096074 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa309cd-0a1d-40cd-8d18-c25d394d2728 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:42.096407 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa309cd-0a1d-40cd-8d18-c25d394d2728 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:42.106666 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa309cd-0a1d-40cd-8d18-c25d394d2728 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 15:47:42.111312 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application [None req-efa309cd-0a1d-40cd-8d18-c25d394d2728 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 30 15:47:42.111312 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:47:42.111312 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:47:42.111312 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:47:42.111312 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:47:42.111312 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:47:42.111312 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:47:42.111312 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:47:42.111312 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:47:42.111312 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:47:42.111312 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:47:42.111312 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:47:42.111312 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 15:47:42.111312 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 15:47:42.111312 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 15:47:42.111312 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 15:47:42.112368 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 15:47:42.112368 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 15:47:42.112368 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 15:47:42.112368 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 15:47:42.112368 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:47:42.112368 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:47:42.112368 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:47:42.112368 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:47:42.112368 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:47:42.112368 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:47:42.112368 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:47:42.112368 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application with Lock( Aug 30 15:47:42.112368 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:47:42.112368 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:47:42.112368 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:47:42.112368 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:47:42.113331 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:47:42.113331 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:47:42.113331 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:47:42.113331 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:47:42.113331 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 15:47:42.113331 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 15:47:42.113331 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 15:47:42.113331 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 15:47:42.113331 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 15:47:42.113331 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 15:47:42.113331 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 30 15:47:42.113331 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Aug 30 15:47:42.113331 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 546/1092] 10.222.0.20 () {56 vars in 1120 bytes} [Tue Aug 30 15:47:41 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 15:47:42.118594 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa309cd-0a1d-40cd-8d18-c25d394d2728 None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:42.137616 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:47:42.158880 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a02a26c-57c8-4f76-829b-8b83b986f76e None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:42.202053 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63545af-6c0a-432c-bd69-01d1ccbd8a78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:42.202673 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63545af-6c0a-432c-bd69-01d1ccbd8a78 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:42.202952 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63545af-6c0a-432c-bd69-01d1ccbd8a78 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:42.203235 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63545af-6c0a-432c-bd69-01d1ccbd8a78 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:42.219788 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa309cd-0a1d-40cd-8d18-c25d394d2728 None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:42.252456 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:47:42.253123 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d485ef1b-f43c-4144-8397-7156055d3f04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:42.253840 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d485ef1b-f43c-4144-8397-7156055d3f04 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:42.254140 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d485ef1b-f43c-4144-8397-7156055d3f04 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:42.254489 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d485ef1b-f43c-4144-8397-7156055d3f04 None admin] PATH_INFO: `/v3/roles` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:42.255139 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d485ef1b-f43c-4144-8397-7156055d3f04 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81512) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:47:42.255442 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d485ef1b-f43c-4144-8397-7156055d3f04 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:42.256897 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d485ef1b-f43c-4144-8397-7156055d3f04 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:42.264986 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 547/1093] 10.222.0.20 () {56 vars in 1115 bytes} [Tue Aug 30 15:47:42 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 15:47:42.270892 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d485ef1b-f43c-4144-8397-7156055d3f04 None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:42.279232 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63545af-6c0a-432c-bd69-01d1ccbd8a78 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:42.280689 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63545af-6c0a-432c-bd69-01d1ccbd8a78 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:42.348282 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:47:42.370156 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d485ef1b-f43c-4144-8397-7156055d3f04 None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:42.379165 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 547/1094] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Aug 30 15:47:42 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 30 15:47:42.385170 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63545af-6c0a-432c-bd69-01d1ccbd8a78 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:42.403856 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7034297f-92d8-463d-8780-5138f6cf08ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:42.404501 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7034297f-92d8-463d-8780-5138f6cf08ff None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:42.404768 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7034297f-92d8-463d-8780-5138f6cf08ff None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:42.405036 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7034297f-92d8-463d-8780-5138f6cf08ff None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:42.417121 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7034297f-92d8-463d-8780-5138f6cf08ff None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:42.418599 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7034297f-92d8-463d-8780-5138f6cf08ff None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:42.433104 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application [None req-7034297f-92d8-463d-8780-5138f6cf08ff None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 15:47:42.433104 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:47:42.433104 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:47:42.433104 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:47:42.433104 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:47:42.433104 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:47:42.433104 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:47:42.433104 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:47:42.433104 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:47:42.433104 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:47:42.433104 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:47:42.433104 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:47:42.433104 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 15:47:42.433104 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 15:47:42.433104 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 15:47:42.433104 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 15:47:42.434312 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:47:42.434312 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:47:42.434312 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:47:42.434312 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:47:42.434312 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:47:42.434312 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:47:42.434312 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:47:42.434312 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application with Lock( Aug 30 15:47:42.434312 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:47:42.434312 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:47:42.434312 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:47:42.434312 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:47:42.434312 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:47:42.434312 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:47:42.434312 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:47:42.434312 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:47:42.434312 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 15:47:42.435224 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 15:47:42.435224 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 15:47:42.435224 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Aug 30 15:47:42.435224 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 548/1095] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 30 15:47:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 15:47:42.440269 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7034297f-92d8-463d-8780-5138f6cf08ff None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:42.458622 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:47:42.483498 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63545af-6c0a-432c-bd69-01d1ccbd8a78 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:42.517535 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d845514b-7745-4484-978a-62064d8030eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:42.518168 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d845514b-7745-4484-978a-62064d8030eb None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:42.518547 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d845514b-7745-4484-978a-62064d8030eb None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:42.518801 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d845514b-7745-4484-978a-62064d8030eb None admin] PATH_INFO: `/v3/projects/1bf9d57f0be14002ba76d32c9b9db4d2` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:42.519989 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d845514b-7745-4484-978a-62064d8030eb None admin] RBAC: Authorizing `identity:get_project(project_id=1bf9d57f0be14002ba76d32c9b9db4d2)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:42.521463 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d845514b-7745-4484-978a-62064d8030eb None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:42.523095 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 548/1096] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Aug 30 15:47:42 2022] GET /identity/v3/projects/1bf9d57f0be14002ba76d32c9b9db4d2 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 15:47:42.529293 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d845514b-7745-4484-978a-62064d8030eb None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:42.541478 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7034297f-92d8-463d-8780-5138f6cf08ff None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:42.569448 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:47:42.576437 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00174c1-0b43-4dc5-88f7-35ea547a0b78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:42.577049 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00174c1-0b43-4dc5-88f7-35ea547a0b78 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:42.577353 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00174c1-0b43-4dc5-88f7-35ea547a0b78 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:42.577609 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00174c1-0b43-4dc5-88f7-35ea547a0b78 None admin] PATH_INFO: `/v3/domains` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:42.578243 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00174c1-0b43-4dc5-88f7-35ea547a0b78 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81512) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:47:42.578543 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00174c1-0b43-4dc5-88f7-35ea547a0b78 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:42.579916 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00174c1-0b43-4dc5-88f7-35ea547a0b78 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:42.591528 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 549/1097] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Aug 30 15:47:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 15:47:42.597261 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00174c1-0b43-4dc5-88f7-35ea547a0b78 None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:42.633331 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d845514b-7745-4484-978a-62064d8030eb None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:42.670026 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5b149d-738e-4eb2-aa1e-cf58b177e47e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:42.670650 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5b149d-738e-4eb2-aa1e-cf58b177e47e None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:42.670908 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5b149d-738e-4eb2-aa1e-cf58b177e47e None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:42.671206 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5b149d-738e-4eb2-aa1e-cf58b177e47e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:42.671922 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5b149d-738e-4eb2-aa1e-cf58b177e47e None admin] RBAC: Adding query filter params (role.id=f265386950a641ac8413b53eda4ae34d, scope.project.id=1bf9d57f0be14002ba76d32c9b9db4d2, user.id=c97a34d0464f4f728681f187e751df14) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:47:42.672224 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5b149d-738e-4eb2-aa1e-cf58b177e47e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:42.673699 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5b149d-738e-4eb2-aa1e-cf58b177e47e None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:42.681254 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 549/1098] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Aug 30 15:47:42 2022] GET /identity/v3/role_assignments?user.id=c97a34d0464f4f728681f187e751df14&scope.project.id=1bf9d57f0be14002ba76d32c9b9db4d2&role.id=f265386950a641ac8413b53eda4ae34d => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 15:47:42.696441 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00174c1-0b43-4dc5-88f7-35ea547a0b78 None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:42.735461 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67423279-9419-4160-8703-e1d7e4d11077 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:42.736114 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67423279-9419-4160-8703-e1d7e4d11077 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:42.736386 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67423279-9419-4160-8703-e1d7e4d11077 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:42.736650 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67423279-9419-4160-8703-e1d7e4d11077 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:42.745263 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67423279-9419-4160-8703-e1d7e4d11077 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:42.746741 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67423279-9419-4160-8703-e1d7e4d11077 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:42.759828 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application [None req-67423279-9419-4160-8703-e1d7e4d11077 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 30 15:47:42.759828 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:47:42.759828 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:47:42.759828 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:47:42.759828 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:47:42.759828 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:47:42.759828 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:47:42.759828 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:47:42.759828 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:47:42.759828 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:47:42.759828 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:47:42.759828 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:47:42.759828 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 15:47:42.759828 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 15:47:42.759828 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 15:47:42.759828 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 15:47:42.761005 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:47:42.761005 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:47:42.761005 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 15:47:42.761005 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 15:47:42.761005 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 15:47:42.761005 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 15:47:42.761005 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:47:42.761005 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:47:42.761005 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:47:42.761005 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:47:42.761005 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:47:42.761005 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application with Lock( Aug 30 15:47:42.761005 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:47:42.761005 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:47:42.761005 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:47:42.761005 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:47:42.761005 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:47:42.761005 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:47:42.762201 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:47:42.762201 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:47:42.762201 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 15:47:42.762201 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 15:47:42.762201 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 15:47:42.762201 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 15:47:42.762201 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 15:47:42.762201 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 15:47:42.762201 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 30 15:47:42.762201 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Aug 30 15:47:42.762201 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 550/1099] 10.222.0.20 () {56 vars in 1093 bytes} [Tue Aug 30 15:47:42 2022] GET /identity/v3/users/nova => generated 82 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 15:47:42.767368 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5b149d-738e-4eb2-aa1e-cf58b177e47e None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:42.776982 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:47:42.876711 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5b149d-738e-4eb2-aa1e-cf58b177e47e None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:42.910264 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65fb918-6b8e-4753-9d92-31cfeb88c2d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:42.910961 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65fb918-6b8e-4753-9d92-31cfeb88c2d5 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:42.911231 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65fb918-6b8e-4753-9d92-31cfeb88c2d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:42.911471 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65fb918-6b8e-4753-9d92-31cfeb88c2d5 None admin] PATH_INFO: `/v3/users` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:42.912207 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65fb918-6b8e-4753-9d92-31cfeb88c2d5 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:47:42.912501 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65fb918-6b8e-4753-9d92-31cfeb88c2d5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:42.913958 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65fb918-6b8e-4753-9d92-31cfeb88c2d5 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:42.931043 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 550/1100] 10.222.0.20 () {56 vars in 1133 bytes} [Tue Aug 30 15:47:42 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:47:42.937394 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67423279-9419-4160-8703-e1d7e4d11077 None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:42.995505 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:47:43.044198 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67423279-9419-4160-8703-e1d7e4d11077 None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:43.076778 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42eccffc-eac1-4467-8810-ba1c8c192f03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:43.077426 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42eccffc-eac1-4467-8810-ba1c8c192f03 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:43.077700 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42eccffc-eac1-4467-8810-ba1c8c192f03 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:43.077971 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42eccffc-eac1-4467-8810-ba1c8c192f03 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:43.143449 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42eccffc-eac1-4467-8810-ba1c8c192f03 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:43.144903 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42eccffc-eac1-4467-8810-ba1c8c192f03 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:43.197559 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:47:43.248621 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 551/1101] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Aug 30 15:47:42 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 30 15:47:43.254565 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65fb918-6b8e-4753-9d92-31cfeb88c2d5 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:43.317926 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:47:43.358464 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65fb918-6b8e-4753-9d92-31cfeb88c2d5 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:43.393510 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a13088-76d4-4c85-8e8b-22b253105026 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:43.394120 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a13088-76d4-4c85-8e8b-22b253105026 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:43.394395 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a13088-76d4-4c85-8e8b-22b253105026 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:43.394662 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a13088-76d4-4c85-8e8b-22b253105026 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:43.404892 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a13088-76d4-4c85-8e8b-22b253105026 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:43.406379 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a13088-76d4-4c85-8e8b-22b253105026 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:43.420854 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-c5a13088-76d4-4c85-8e8b-22b253105026 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 15:47:43.420854 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:47:43.420854 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:47:43.420854 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:47:43.420854 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:47:43.420854 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:47:43.420854 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:47:43.420854 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:47:43.420854 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:47:43.420854 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:47:43.420854 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:47:43.420854 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:47:43.420854 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 15:47:43.420854 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 15:47:43.420854 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 15:47:43.420854 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 15:47:43.421812 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:47:43.421812 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:47:43.421812 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:47:43.421812 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:47:43.421812 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:47:43.421812 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:47:43.421812 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:47:43.421812 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Aug 30 15:47:43.421812 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:47:43.421812 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:47:43.421812 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:47:43.421812 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:47:43.421812 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:47:43.421812 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:47:43.421812 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:47:43.421812 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:47:43.421812 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 15:47:43.422719 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 15:47:43.422719 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 15:47:43.422719 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Aug 30 15:47:43.422719 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 551/1102] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 30 15:47:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 15:47:43.427979 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42eccffc-eac1-4467-8810-ba1c8c192f03 None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:43.530555 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42eccffc-eac1-4467-8810-ba1c8c192f03 None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:43.565459 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf59d1f-0bb0-47aa-9e77-c7e600ace558 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:43.566057 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf59d1f-0bb0-47aa-9e77-c7e600ace558 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:43.566326 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf59d1f-0bb0-47aa-9e77-c7e600ace558 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:43.566588 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf59d1f-0bb0-47aa-9e77-c7e600ace558 None admin] PATH_INFO: `/v3/domains` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:43.567218 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf59d1f-0bb0-47aa-9e77-c7e600ace558 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81512) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:47:43.567507 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf59d1f-0bb0-47aa-9e77-c7e600ace558 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:43.568862 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf59d1f-0bb0-47aa-9e77-c7e600ace558 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:43.580125 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 552/1103] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Aug 30 15:47:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:47:43.586109 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a13088-76d4-4c85-8e8b-22b253105026 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:43.692908 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a13088-76d4-4c85-8e8b-22b253105026 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:43.709352 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:47:43.727545 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d74c593-9b2f-40c5-970d-c7ac47445982 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:43.728171 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d74c593-9b2f-40c5-970d-c7ac47445982 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:43.728451 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d74c593-9b2f-40c5-970d-c7ac47445982 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:43.728717 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d74c593-9b2f-40c5-970d-c7ac47445982 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:43.735401 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d74c593-9b2f-40c5-970d-c7ac47445982 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:43.736861 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d74c593-9b2f-40c5-970d-c7ac47445982 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:43.748295 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-1d74c593-9b2f-40c5-970d-c7ac47445982 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 15:47:43.748295 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:47:43.748295 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:47:43.748295 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:47:43.748295 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:47:43.748295 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:47:43.748295 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:47:43.748295 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:47:43.748295 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:47:43.748295 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:47:43.748295 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:47:43.748295 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:47:43.748295 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 15:47:43.748295 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 15:47:43.748295 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 15:47:43.754230 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 15:47:43.754230 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:47:43.754230 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:47:43.754230 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:47:43.754230 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:47:43.754230 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:47:43.754230 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:47:43.754230 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:47:43.754230 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Aug 30 15:47:43.754230 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:47:43.754230 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:47:43.754230 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:47:43.754230 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:47:43.754230 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:47:43.754230 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:47:43.754230 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:47:43.754230 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:47:43.755413 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 15:47:43.755413 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 15:47:43.755413 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 15:47:43.755413 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 15:47:43.755413 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 15:47:43.755413 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 15:47:43.755413 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 15:47:43.755413 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Aug 30 15:47:43.755413 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 552/1104] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Aug 30 15:47:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 15:47:43.756022 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf59d1f-0bb0-47aa-9e77-c7e600ace558 None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:43.769189 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:47:43.860833 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf59d1f-0bb0-47aa-9e77-c7e600ace558 None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:43.894196 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43ec6cb-1d51-4cdd-afef-2a0391c8b19e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:43.894797 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43ec6cb-1d51-4cdd-afef-2a0391c8b19e None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:43.895077 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43ec6cb-1d51-4cdd-afef-2a0391c8b19e None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:43.895348 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43ec6cb-1d51-4cdd-afef-2a0391c8b19e None admin] PATH_INFO: `/v3/projects` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:43.896043 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43ec6cb-1d51-4cdd-afef-2a0391c8b19e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81512) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:47:43.896348 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43ec6cb-1d51-4cdd-afef-2a0391c8b19e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:43.897827 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43ec6cb-1d51-4cdd-afef-2a0391c8b19e None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:43.909543 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 553/1105] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Aug 30 15:47:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 15:47:43.915722 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d74c593-9b2f-40c5-970d-c7ac47445982 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:44.026090 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d74c593-9b2f-40c5-970d-c7ac47445982 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:44.061005 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0defec3-c91c-48c6-b86a-064006cb7c85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:44.061636 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0defec3-c91c-48c6-b86a-064006cb7c85 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:44.061913 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0defec3-c91c-48c6-b86a-064006cb7c85 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:44.062178 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0defec3-c91c-48c6-b86a-064006cb7c85 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:44.062902 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0defec3-c91c-48c6-b86a-064006cb7c85 None admin] RBAC: Adding query filter params (role.id=a89e0730512d449ca32479022aaf58d1, scope.project.id=854fe71b49224dff9a4b1766a20ab266, user.id=7a1e756ae9be4a7bac1e81acf07e8f3b) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:47:44.063206 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0defec3-c91c-48c6-b86a-064006cb7c85 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:44.064636 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0defec3-c91c-48c6-b86a-064006cb7c85 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:44.081241 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 553/1106] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Aug 30 15:47:43 2022] GET /identity/v3/role_assignments?user.id=7a1e756ae9be4a7bac1e81acf07e8f3b&scope.project.id=854fe71b49224dff9a4b1766a20ab266&role.id=a89e0730512d449ca32479022aaf58d1 => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:47:44.603457 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009e8304-6df0-4ba2-b5a0-252c071d9b54 None None] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:44.603998 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009e8304-6df0-4ba2-b5a0-252c071d9b54 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:44.603998 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009e8304-6df0-4ba2-b5a0-252c071d9b54 None None] PATH_INFO: `/` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:44.605368 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 554/1107] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 30 15:47:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 15:47:44.611097 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e98a45a-5742-4e38-bc32-299ac1bfed73 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:44.611552 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e98a45a-5742-4e38-bc32-299ac1bfed73 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:44.611639 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e98a45a-5742-4e38-bc32-299ac1bfed73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:44.633963 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-8e98a45a-5742-4e38-bc32-299ac1bfed73 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:47:44.635152 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-8e98a45a-5742-4e38-bc32-299ac1bfed73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:47:44.672978 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:47:44.707583 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 554/1108] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 30 15:47:44 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 15:47:44.972865 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009e8304-6df0-4ba2-b5a0-252c071d9b54 None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:45.058333 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:47:45.091351 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009e8304-6df0-4ba2-b5a0-252c071d9b54 None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:45.127178 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95277aa-2e35-4111-9db9-61d98d68ac6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:45.127794 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95277aa-2e35-4111-9db9-61d98d68ac6b None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:45.128042 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95277aa-2e35-4111-9db9-61d98d68ac6b None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:45.128345 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95277aa-2e35-4111-9db9-61d98d68ac6b None admin] PATH_INFO: `/v3/domains/default` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:45.129593 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95277aa-2e35-4111-9db9-61d98d68ac6b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:45.131035 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95277aa-2e35-4111-9db9-61d98d68ac6b None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:45.132610 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 555/1109] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 30 15:47:44 2022] GET /identity/v3/domains/default => generated 199 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:47:45.138909 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e98a45a-5742-4e38-bc32-299ac1bfed73 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:45.247523 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e98a45a-5742-4e38-bc32-299ac1bfed73 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:45.282544 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b110d38-b430-4a25-9a61-0f386f73f2a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:45.283140 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b110d38-b430-4a25-9a61-0f386f73f2a9 None admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:45.283416 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b110d38-b430-4a25-9a61-0f386f73f2a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:45.283683 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b110d38-b430-4a25-9a61-0f386f73f2a9 None admin] PATH_INFO: `/v3/users` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:45.284331 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b110d38-b430-4a25-9a61-0f386f73f2a9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:45.285816 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b110d38-b430-4a25-9a61-0f386f73f2a9 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:45.314196 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 555/1110] 10.222.0.20 () {60 vars in 1132 bytes} [Tue Aug 30 15:47:45 2022] POST /identity/v3/users => generated 286 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 15:47:47.234188 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a8deb3-86d1-41dd-b71e-f2e7c820bcbd None None] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:47.235049 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a8deb3-86d1-41dd-b71e-f2e7c820bcbd None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:47.235049 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a8deb3-86d1-41dd-b71e-f2e7c820bcbd None None] PATH_INFO: `/` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:47.237528 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 556/1111] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 30 15:47:47 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 15:47:47.246252 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b6cc42-e9e9-44a6-9bb1-135b636de02f None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:47.246920 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b6cc42-e9e9-44a6-9bb1-135b636de02f None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:47.247175 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b6cc42-e9e9-44a6-9bb1-135b636de02f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:47.268111 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-69b6cc42-e9e9-44a6-9bb1-135b636de02f None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:47:47.269319 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-69b6cc42-e9e9-44a6-9bb1-135b636de02f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:47:47.291252 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:47:47.345897 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 556/1112] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 30 15:47:47 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 15:47:47.623158 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a8deb3-86d1-41dd-b71e-f2e7c820bcbd None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:47.738483 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a8deb3-86d1-41dd-b71e-f2e7c820bcbd None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:47.775297 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd24e3d9-e3a2-49c6-ab38-afd1df893f7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:47.775917 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd24e3d9-e3a2-49c6-ab38-afd1df893f7c None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:47.776189 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd24e3d9-e3a2-49c6-ab38-afd1df893f7c None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:47.776456 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd24e3d9-e3a2-49c6-ab38-afd1df893f7c None admin] PATH_INFO: `/v3/roles/6b4385c3b449456d848c9abb16766d25` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:47.777999 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd24e3d9-e3a2-49c6-ab38-afd1df893f7c None admin] RBAC: Authorizing `identity:get_role(role_id=6b4385c3b449456d848c9abb16766d25)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:47.779377 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd24e3d9-e3a2-49c6-ab38-afd1df893f7c None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:47.780700 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 557/1113] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Aug 30 15:47:47 2022] GET /identity/v3/roles/6b4385c3b449456d848c9abb16766d25 => generated 223 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:47:47.786784 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b6cc42-e9e9-44a6-9bb1-135b636de02f None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:47.887451 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b6cc42-e9e9-44a6-9bb1-135b636de02f None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:47.924170 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1a0bc0-9607-4250-980a-be827c92d54e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:47.924808 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1a0bc0-9607-4250-980a-be827c92d54e None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:47.925080 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1a0bc0-9607-4250-980a-be827c92d54e None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:47.925382 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1a0bc0-9607-4250-980a-be827c92d54e None admin] PATH_INFO: `/v3/users/a772722563694bd0bd819f81a576860d` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:47.938829 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1a0bc0-9607-4250-980a-be827c92d54e None admin] RBAC: Authorizing `identity:get_user(user_id=a772722563694bd0bd819f81a576860d)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:47.940274 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1a0bc0-9607-4250-980a-be827c92d54e None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:47.941880 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 557/1114] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Aug 30 15:47:47 2022] GET /identity/v3/users/a772722563694bd0bd819f81a576860d => generated 286 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:47:47.948321 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd24e3d9-e3a2-49c6-ab38-afd1df893f7c None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:47.962649 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:47:48.072417 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd24e3d9-e3a2-49c6-ab38-afd1df893f7c None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:48.109320 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2c570d-513a-4bb7-9c01-ed2ab1b3650e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:48.109936 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2c570d-513a-4bb7-9c01-ed2ab1b3650e None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:48.110221 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2c570d-513a-4bb7-9c01-ed2ab1b3650e None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:48.110489 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2c570d-513a-4bb7-9c01-ed2ab1b3650e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:48.183208 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2c570d-513a-4bb7-9c01-ed2ab1b3650e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:48.184632 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2c570d-513a-4bb7-9c01-ed2ab1b3650e None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:48.193184 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:47:48.300845 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 558/1115] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Aug 30 15:47:47 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 30 15:47:48.307197 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1a0bc0-9607-4250-980a-be827c92d54e None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:48.325541 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:47:48.412540 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1a0bc0-9607-4250-980a-be827c92d54e None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:48.447847 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deaad7ac-f11b-4a92-8508-e8e9efd2b58b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:48.448399 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deaad7ac-f11b-4a92-8508-e8e9efd2b58b None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:48.448668 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deaad7ac-f11b-4a92-8508-e8e9efd2b58b None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:48.448931 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deaad7ac-f11b-4a92-8508-e8e9efd2b58b None admin] PATH_INFO: `/v3/projects/1bf9d57f0be14002ba76d32c9b9db4d2` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:48.450113 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deaad7ac-f11b-4a92-8508-e8e9efd2b58b None admin] RBAC: Authorizing `identity:get_project(project_id=1bf9d57f0be14002ba76d32c9b9db4d2)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:48.451607 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deaad7ac-f11b-4a92-8508-e8e9efd2b58b None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:48.453258 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 558/1116] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Aug 30 15:47:48 2022] GET /identity/v3/projects/1bf9d57f0be14002ba76d32c9b9db4d2 => generated 311 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:47:48.460239 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2c570d-513a-4bb7-9c01-ed2ab1b3650e None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:48.570443 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2c570d-513a-4bb7-9c01-ed2ab1b3650e None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:48.577710 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:47:48.606264 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3477acc5-aae0-4759-95f0-e4501fef83f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:48.607061 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3477acc5-aae0-4759-95f0-e4501fef83f6 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:48.607337 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3477acc5-aae0-4759-95f0-e4501fef83f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:48.607604 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3477acc5-aae0-4759-95f0-e4501fef83f6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:48.608318 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3477acc5-aae0-4759-95f0-e4501fef83f6 None admin] RBAC: Adding query filter params (role.id=6b4385c3b449456d848c9abb16766d25, scope.project.id=1bf9d57f0be14002ba76d32c9b9db4d2, user.id=a772722563694bd0bd819f81a576860d) {{(pid=81512) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:47:48.608627 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3477acc5-aae0-4759-95f0-e4501fef83f6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:48.610083 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3477acc5-aae0-4759-95f0-e4501fef83f6 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:48.615409 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 559/1117] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Aug 30 15:47:48 2022] GET /identity/v3/role_assignments?user.id=a772722563694bd0bd819f81a576860d&scope.project.id=1bf9d57f0be14002ba76d32c9b9db4d2&role.id=6b4385c3b449456d848c9abb16766d25 => generated 259 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:47:50.498167 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de952a8-968e-4721-b2c8-e2b9830bc6d7 None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:50.498857 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de952a8-968e-4721-b2c8-e2b9830bc6d7 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:50.498857 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de952a8-968e-4721-b2c8-e2b9830bc6d7 None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:50.500608 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 559/1118] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 30 15:47:50 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 15:47:50.508379 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acfeb7b-92b3-4397-8935-20c107461d4b None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:50.508802 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acfeb7b-92b3-4397-8935-20c107461d4b None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:50.509090 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acfeb7b-92b3-4397-8935-20c107461d4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:50.536378 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-2acfeb7b-92b3-4397-8935-20c107461d4b None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:47:50.537696 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-2acfeb7b-92b3-4397-8935-20c107461d4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:47:50.606132 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 560/1119] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 30 15:47:50 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 15:47:50.872032 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de952a8-968e-4721-b2c8-e2b9830bc6d7 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:50.967139 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:47:50.988646 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de952a8-968e-4721-b2c8-e2b9830bc6d7 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:51.024086 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a342d4-f737-455e-b6bb-efe938ef1883 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:51.024701 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a342d4-f737-455e-b6bb-efe938ef1883 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:51.024991 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a342d4-f737-455e-b6bb-efe938ef1883 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:51.025246 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a342d4-f737-455e-b6bb-efe938ef1883 None admin] PATH_INFO: `/v3/roles/6b4385c3b449456d848c9abb16766d25` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:51.026743 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a342d4-f737-455e-b6bb-efe938ef1883 None admin] RBAC: Authorizing `identity:get_role(role_id=6b4385c3b449456d848c9abb16766d25)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:51.028115 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a342d4-f737-455e-b6bb-efe938ef1883 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:51.029501 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 560/1120] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Aug 30 15:47:50 2022] GET /identity/v3/roles/6b4385c3b449456d848c9abb16766d25 => generated 223 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:47:51.035866 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acfeb7b-92b3-4397-8935-20c107461d4b None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:51.137593 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:47:51.140575 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acfeb7b-92b3-4397-8935-20c107461d4b None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:51.175408 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5cfec9-7d86-4081-b869-f539d330ab07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:51.176020 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5cfec9-7d86-4081-b869-f539d330ab07 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:51.176290 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5cfec9-7d86-4081-b869-f539d330ab07 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:51.176559 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5cfec9-7d86-4081-b869-f539d330ab07 None admin] PATH_INFO: `/v3/users/a772722563694bd0bd819f81a576860d` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:51.177727 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5cfec9-7d86-4081-b869-f539d330ab07 None admin] RBAC: Authorizing `identity:get_user(user_id=a772722563694bd0bd819f81a576860d)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:51.179160 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5cfec9-7d86-4081-b869-f539d330ab07 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:51.180724 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 561/1121] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Aug 30 15:47:51 2022] GET /identity/v3/users/a772722563694bd0bd819f81a576860d => generated 286 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:47:51.187230 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a342d4-f737-455e-b6bb-efe938ef1883 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:51.312340 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a342d4-f737-455e-b6bb-efe938ef1883 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:51.348602 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c54c4a-9008-41fe-8b2c-d94cdd7b1a1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:51.349247 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c54c4a-9008-41fe-8b2c-d94cdd7b1a1a None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:51.349522 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c54c4a-9008-41fe-8b2c-d94cdd7b1a1a None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:51.349787 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c54c4a-9008-41fe-8b2c-d94cdd7b1a1a None admin] PATH_INFO: `/v3/projects/1bf9d57f0be14002ba76d32c9b9db4d2` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:51.350930 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c54c4a-9008-41fe-8b2c-d94cdd7b1a1a None admin] RBAC: Authorizing `identity:get_project(project_id=1bf9d57f0be14002ba76d32c9b9db4d2)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:51.352355 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c54c4a-9008-41fe-8b2c-d94cdd7b1a1a None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:51.353993 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 561/1122] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Aug 30 15:47:51 2022] GET /identity/v3/projects/1bf9d57f0be14002ba76d32c9b9db4d2 => generated 311 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:47:51.360126 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5cfec9-7d86-4081-b869-f539d330ab07 None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:51.461199 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5cfec9-7d86-4081-b869-f539d330ab07 None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:51.497753 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2678dc62-db9e-4161-a3a6-89a0c2f5e378 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:51.498377 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2678dc62-db9e-4161-a3a6-89a0c2f5e378 None admin] REQUEST_METHOD: `PUT` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:51.498651 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2678dc62-db9e-4161-a3a6-89a0c2f5e378 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:51.498917 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2678dc62-db9e-4161-a3a6-89a0c2f5e378 None admin] PATH_INFO: `/v3/projects/1bf9d57f0be14002ba76d32c9b9db4d2/users/a772722563694bd0bd819f81a576860d/roles/6b4385c3b449456d848c9abb16766d25` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:51.501088 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2678dc62-db9e-4161-a3a6-89a0c2f5e378 None admin] RBAC: Authorizing `identity:create_grant(project_id=1bf9d57f0be14002ba76d32c9b9db4d2, user_id=a772722563694bd0bd819f81a576860d, role_id=6b4385c3b449456d848c9abb16766d25)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:51.502542 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2678dc62-db9e-4161-a3a6-89a0c2f5e378 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:51.519846 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 562/1123] 10.222.0.20 () {58 vars in 1439 bytes} [Tue Aug 30 15:47:51 2022] PUT /identity/v3/projects/1bf9d57f0be14002ba76d32c9b9db4d2/users/a772722563694bd0bd819f81a576860d/roles/6b4385c3b449456d848c9abb16766d25 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:47:53.397858 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea83d1b-3a7d-479a-a066-50d83c0fe870 None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:53.398443 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea83d1b-3a7d-479a-a066-50d83c0fe870 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:53.398443 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea83d1b-3a7d-479a-a066-50d83c0fe870 None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:53.399863 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 562/1124] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 30 15:47:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 15:47:53.405179 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2862e124-8b03-499f-b1a0-816b31cc25ce None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:53.405599 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2862e124-8b03-499f-b1a0-816b31cc25ce None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:53.405785 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2862e124-8b03-499f-b1a0-816b31cc25ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:53.424275 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-2862e124-8b03-499f-b1a0-816b31cc25ce None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:47:53.425511 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-2862e124-8b03-499f-b1a0-816b31cc25ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:47:53.448945 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:47:53.497736 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 563/1125] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 30 15:47:53 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 30 15:47:53.762092 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea83d1b-3a7d-479a-a066-50d83c0fe870 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:53.784404 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:47:53.877110 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea83d1b-3a7d-479a-a066-50d83c0fe870 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:53.916191 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e37203f-24c7-4a7d-bc3f-214741d23772 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:53.916829 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e37203f-24c7-4a7d-bc3f-214741d23772 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:53.917108 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e37203f-24c7-4a7d-bc3f-214741d23772 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:53.917403 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e37203f-24c7-4a7d-bc3f-214741d23772 None admin] PATH_INFO: `/v3/roles/6b4385c3b449456d848c9abb16766d25` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:53.918920 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e37203f-24c7-4a7d-bc3f-214741d23772 None admin] RBAC: Authorizing `identity:get_role(role_id=6b4385c3b449456d848c9abb16766d25)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:53.920307 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e37203f-24c7-4a7d-bc3f-214741d23772 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:53.921688 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 563/1126] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Aug 30 15:47:53 2022] GET /identity/v3/roles/6b4385c3b449456d848c9abb16766d25 => generated 223 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:47:53.928472 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2862e124-8b03-499f-b1a0-816b31cc25ce None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:54.061392 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2862e124-8b03-499f-b1a0-816b31cc25ce None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:54.097738 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae94443e-ca61-4cd0-b3a6-23de5327c877 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:54.098359 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae94443e-ca61-4cd0-b3a6-23de5327c877 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:54.098637 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae94443e-ca61-4cd0-b3a6-23de5327c877 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:54.098903 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae94443e-ca61-4cd0-b3a6-23de5327c877 None admin] PATH_INFO: `/v3/users/a772722563694bd0bd819f81a576860d` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:54.100157 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae94443e-ca61-4cd0-b3a6-23de5327c877 None admin] RBAC: Authorizing `identity:get_user(user_id=a772722563694bd0bd819f81a576860d)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:54.101635 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae94443e-ca61-4cd0-b3a6-23de5327c877 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:54.103248 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 564/1127] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Aug 30 15:47:53 2022] GET /identity/v3/users/a772722563694bd0bd819f81a576860d => generated 286 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:47:54.112762 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e37203f-24c7-4a7d-bc3f-214741d23772 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:54.203663 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:47:54.219837 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e37203f-24c7-4a7d-bc3f-214741d23772 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:54.258305 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec80828-5925-4fe2-989b-af9e12f7337f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:54.258905 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec80828-5925-4fe2-989b-af9e12f7337f None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:54.259178 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec80828-5925-4fe2-989b-af9e12f7337f None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:54.259444 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec80828-5925-4fe2-989b-af9e12f7337f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:54.334397 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec80828-5925-4fe2-989b-af9e12f7337f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:54.336046 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec80828-5925-4fe2-989b-af9e12f7337f None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:54.424786 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:47:54.441616 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 564/1128] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Aug 30 15:47:54 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 30 15:47:54.448029 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae94443e-ca61-4cd0-b3a6-23de5327c877 None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:54.477974 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:47:54.562207 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae94443e-ca61-4cd0-b3a6-23de5327c877 None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:54.597594 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f86709-b808-4400-ab89-4b82ba2f5fc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:54.598143 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f86709-b808-4400-ab89-4b82ba2f5fc7 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:54.598388 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f86709-b808-4400-ab89-4b82ba2f5fc7 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:54.598685 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f86709-b808-4400-ab89-4b82ba2f5fc7 None admin] PATH_INFO: `/v3/projects/1bf9d57f0be14002ba76d32c9b9db4d2` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:54.599882 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f86709-b808-4400-ab89-4b82ba2f5fc7 None admin] RBAC: Authorizing `identity:get_project(project_id=1bf9d57f0be14002ba76d32c9b9db4d2)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:54.601347 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f86709-b808-4400-ab89-4b82ba2f5fc7 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:54.603058 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 565/1129] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Aug 30 15:47:54 2022] GET /identity/v3/projects/1bf9d57f0be14002ba76d32c9b9db4d2 => generated 311 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:47:54.609649 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec80828-5925-4fe2-989b-af9e12f7337f None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:54.710053 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec80828-5925-4fe2-989b-af9e12f7337f None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:54.751631 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02a4d5d-067f-458e-860b-e347ea8c9522 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:54.752232 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02a4d5d-067f-458e-860b-e347ea8c9522 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:54.752504 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02a4d5d-067f-458e-860b-e347ea8c9522 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:54.752764 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02a4d5d-067f-458e-860b-e347ea8c9522 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:54.753514 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02a4d5d-067f-458e-860b-e347ea8c9522 None admin] RBAC: Adding query filter params (role.id=6b4385c3b449456d848c9abb16766d25, scope.project.id=1bf9d57f0be14002ba76d32c9b9db4d2, user.id=a772722563694bd0bd819f81a576860d) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:47:54.753813 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02a4d5d-067f-458e-860b-e347ea8c9522 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:54.755241 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02a4d5d-067f-458e-860b-e347ea8c9522 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:54.761365 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 565/1130] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Aug 30 15:47:54 2022] GET /identity/v3/role_assignments?user.id=a772722563694bd0bd819f81a576860d&scope.project.id=1bf9d57f0be14002ba76d32c9b9db4d2&role.id=6b4385c3b449456d848c9abb16766d25 => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:47:56.641883 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6dde96-d566-4e7e-8598-52b18eddff04 None None] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:56.642722 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6dde96-d566-4e7e-8598-52b18eddff04 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:56.642722 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6dde96-d566-4e7e-8598-52b18eddff04 None None] PATH_INFO: `/` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:56.644853 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 566/1131] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 30 15:47:56 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 15:47:56.653327 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ba5724-3342-4077-896b-4f85d26b380f None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:56.653748 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ba5724-3342-4077-896b-4f85d26b380f None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:56.653894 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ba5724-3342-4077-896b-4f85d26b380f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:56.679237 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-47ba5724-3342-4077-896b-4f85d26b380f None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:47:56.680421 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-47ba5724-3342-4077-896b-4f85d26b380f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:47:56.739095 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:47:56.764487 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 566/1132] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 30 15:47:56 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 15:47:57.028024 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6dde96-d566-4e7e-8598-52b18eddff04 None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:57.139167 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:47:57.147349 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6dde96-d566-4e7e-8598-52b18eddff04 None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:57.179872 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757f1328-2445-4145-a70c-9b01e91a40af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:57.180486 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757f1328-2445-4145-a70c-9b01e91a40af None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:57.180758 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757f1328-2445-4145-a70c-9b01e91a40af None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:57.181023 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757f1328-2445-4145-a70c-9b01e91a40af None admin] PATH_INFO: `/v3/domains/default` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:57.182142 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757f1328-2445-4145-a70c-9b01e91a40af None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:57.183562 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757f1328-2445-4145-a70c-9b01e91a40af None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:57.185271 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 567/1133] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 30 15:47:57 2022] GET /identity/v3/domains/default => generated 199 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:47:57.191552 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ba5724-3342-4077-896b-4f85d26b380f None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:57.290777 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ba5724-3342-4077-896b-4f85d26b380f None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:57.325390 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fb34e8-e723-4654-b183-14c2ed55cdf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:57.325980 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fb34e8-e723-4654-b183-14c2ed55cdf9 None admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:57.326252 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fb34e8-e723-4654-b183-14c2ed55cdf9 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:57.326514 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fb34e8-e723-4654-b183-14c2ed55cdf9 None admin] PATH_INFO: `/v3/projects` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:57.327172 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7fb34e8-e723-4654-b183-14c2ed55cdf9 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:57.328597 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7fb34e8-e723-4654-b183-14c2ed55cdf9 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:57.360408 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 567/1134] 10.222.0.20 () {60 vars in 1141 bytes} [Tue Aug 30 15:47:57 2022] POST /identity/v3/projects => generated 311 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 15:47:59.258954 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8b0dc4-315a-4747-bc8c-4acf5b8b8a47 None None] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:59.259825 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8b0dc4-315a-4747-bc8c-4acf5b8b8a47 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:59.259825 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8b0dc4-315a-4747-bc8c-4acf5b8b8a47 None None] PATH_INFO: `/` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:59.262036 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 568/1135] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 30 15:47:59 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 15:47:59.267453 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d5c642-a2cb-4eff-8a08-bb292ebe8c4d None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:59.267875 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d5c642-a2cb-4eff-8a08-bb292ebe8c4d None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:59.268102 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d5c642-a2cb-4eff-8a08-bb292ebe8c4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:59.287788 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-50d5c642-a2cb-4eff-8a08-bb292ebe8c4d None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:47:59.288947 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-50d5c642-a2cb-4eff-8a08-bb292ebe8c4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:47:59.317760 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:47:59.365208 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 568/1136] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 30 15:47:59 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 15:47:59.641138 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8b0dc4-315a-4747-bc8c-4acf5b8b8a47 None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:59.749498 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8b0dc4-315a-4747-bc8c-4acf5b8b8a47 None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:59.786858 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3d378e-1875-427c-b16e-9f59cd8de944 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:59.787464 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3d378e-1875-427c-b16e-9f59cd8de944 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:59.787739 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3d378e-1875-427c-b16e-9f59cd8de944 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:59.788002 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3d378e-1875-427c-b16e-9f59cd8de944 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:59.789263 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3d378e-1875-427c-b16e-9f59cd8de944 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:59.790691 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3d378e-1875-427c-b16e-9f59cd8de944 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:59.792243 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 569/1137] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 30 15:47:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:47:59.799267 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d5c642-a2cb-4eff-8a08-bb292ebe8c4d None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:47:59.910403 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d5c642-a2cb-4eff-8a08-bb292ebe8c4d None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:47:59.948045 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311bb34b-b0dd-4cf6-a43f-4f023f8086fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:47:59.948650 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311bb34b-b0dd-4cf6-a43f-4f023f8086fb None admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:47:59.948925 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311bb34b-b0dd-4cf6-a43f-4f023f8086fb None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:47:59.949196 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311bb34b-b0dd-4cf6-a43f-4f023f8086fb None admin] PATH_INFO: `/v3/users` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:47:59.949877 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311bb34b-b0dd-4cf6-a43f-4f023f8086fb None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:47:59.951339 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311bb34b-b0dd-4cf6-a43f-4f023f8086fb None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:47:59.977315 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 569/1138] 10.222.0.20 () {60 vars in 1132 bytes} [Tue Aug 30 15:47:59 2022] POST /identity/v3/users => generated 286 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 15:48:01.908921 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f692db88-ac29-44ca-968e-d6e7ea404d7d None None] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:48:01.909849 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f692db88-ac29-44ca-968e-d6e7ea404d7d None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:48:01.909849 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f692db88-ac29-44ca-968e-d6e7ea404d7d None None] PATH_INFO: `/` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:48:01.911813 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 570/1139] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 30 15:48:01 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 15:48:01.918206 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5196f16e-2019-4259-b57a-5fd2116246c0 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:48:01.919001 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5196f16e-2019-4259-b57a-5fd2116246c0 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:48:01.919090 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5196f16e-2019-4259-b57a-5fd2116246c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:48:01.939196 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-5196f16e-2019-4259-b57a-5fd2116246c0 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:48:01.940393 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-5196f16e-2019-4259-b57a-5fd2116246c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:48:01.955086 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:48:02.015438 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 570/1140] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 30 15:48:01 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 15:48:02.281792 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f692db88-ac29-44ca-968e-d6e7ea404d7d None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:48:02.323088 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:48:02.403673 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f692db88-ac29-44ca-968e-d6e7ea404d7d None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:48:02.438761 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11b989e-cd98-4067-8f69-efb710dbf316 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:48:02.439396 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11b989e-cd98-4067-8f69-efb710dbf316 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:48:02.439648 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11b989e-cd98-4067-8f69-efb710dbf316 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:48:02.439936 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11b989e-cd98-4067-8f69-efb710dbf316 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:48:02.451259 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11b989e-cd98-4067-8f69-efb710dbf316 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 15:48:02.456036 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application [None req-f11b989e-cd98-4067-8f69-efb710dbf316 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 15:48:02.456036 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:48:02.456036 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:48:02.456036 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:48:02.456036 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:48:02.456036 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:48:02.456036 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:48:02.456036 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:48:02.456036 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:48:02.456036 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:48:02.456036 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:48:02.456036 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:48:02.456036 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 15:48:02.456036 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 15:48:02.456036 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 15:48:02.456036 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 15:48:02.457192 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 15:48:02.457192 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 15:48:02.457192 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 15:48:02.457192 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 15:48:02.457192 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:48:02.457192 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:48:02.457192 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:48:02.457192 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:48:02.457192 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:48:02.457192 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:48:02.457192 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:48:02.457192 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application with Lock( Aug 30 15:48:02.457192 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:48:02.457192 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:48:02.457192 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:48:02.457192 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:48:02.458376 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:48:02.458376 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:48:02.458376 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:48:02.458376 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:48:02.458376 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 15:48:02.458376 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 15:48:02.458376 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 15:48:02.458376 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 15:48:02.458376 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 15:48:02.458376 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 15:48:02.458376 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 15:48:02.458376 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Aug 30 15:48:02.458376 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 571/1141] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Aug 30 15:48:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 15:48:02.463950 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5196f16e-2019-4259-b57a-5fd2116246c0 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:48:02.575301 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5196f16e-2019-4259-b57a-5fd2116246c0 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:48:02.610585 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c182ed-cfdd-4a98-8666-aae1b4a53597 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:48:02.611188 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c182ed-cfdd-4a98-8666-aae1b4a53597 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:48:02.611446 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c182ed-cfdd-4a98-8666-aae1b4a53597 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:48:02.611706 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c182ed-cfdd-4a98-8666-aae1b4a53597 None admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:48:02.612538 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c182ed-cfdd-4a98-8666-aae1b4a53597 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:48:02.612828 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c182ed-cfdd-4a98-8666-aae1b4a53597 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:48:02.614214 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c182ed-cfdd-4a98-8666-aae1b4a53597 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:48:02.622550 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 571/1142] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Aug 30 15:48:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:48:02.631308 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11b989e-cd98-4067-8f69-efb710dbf316 None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:48:02.732391 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:48:02.747427 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11b989e-cd98-4067-8f69-efb710dbf316 None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:48:02.784056 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fdda29-375d-4336-be39-434612df7bea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:48:02.784664 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fdda29-375d-4336-be39-434612df7bea None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:48:02.784933 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fdda29-375d-4336-be39-434612df7bea None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:48:02.785161 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fdda29-375d-4336-be39-434612df7bea None admin] PATH_INFO: `/v3/users/edf7c685b37e429595b393c633fed567` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:48:02.798732 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fdda29-375d-4336-be39-434612df7bea None admin] RBAC: Authorizing `identity:get_user(user_id=edf7c685b37e429595b393c633fed567)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:48:02.800270 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fdda29-375d-4336-be39-434612df7bea None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:48:02.801889 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 572/1143] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Aug 30 15:48:02 2022] GET /identity/v3/users/edf7c685b37e429595b393c633fed567 => generated 286 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:48:02.808160 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c182ed-cfdd-4a98-8666-aae1b4a53597 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:48:02.826606 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:48:02.935522 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c182ed-cfdd-4a98-8666-aae1b4a53597 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:48:02.969937 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31290f60-af06-4562-8a73-bb4e1661643c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:48:02.970541 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31290f60-af06-4562-8a73-bb4e1661643c None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:48:02.970816 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31290f60-af06-4562-8a73-bb4e1661643c None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:48:02.971085 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31290f60-af06-4562-8a73-bb4e1661643c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:48:03.039778 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31290f60-af06-4562-8a73-bb4e1661643c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:48:03.041243 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31290f60-af06-4562-8a73-bb4e1661643c None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:48:03.053841 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:48:03.143067 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 572/1144] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Aug 30 15:48:02 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 30 15:48:03.150070 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fdda29-375d-4336-be39-434612df7bea None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:48:03.254802 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fdda29-375d-4336-be39-434612df7bea None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:48:03.289898 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33721e94-3e10-499d-a987-3a97bd944a94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:48:03.290516 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33721e94-3e10-499d-a987-3a97bd944a94 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:48:03.290789 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33721e94-3e10-499d-a987-3a97bd944a94 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:48:03.291055 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33721e94-3e10-499d-a987-3a97bd944a94 None admin] PATH_INFO: `/v3/projects/0082e220ecce4c6ab75b86249985f4dd` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:48:03.292266 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33721e94-3e10-499d-a987-3a97bd944a94 None admin] RBAC: Authorizing `identity:get_project(project_id=0082e220ecce4c6ab75b86249985f4dd)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:48:03.293809 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33721e94-3e10-499d-a987-3a97bd944a94 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:48:03.295401 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 573/1145] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Aug 30 15:48:03 2022] GET /identity/v3/projects/0082e220ecce4c6ab75b86249985f4dd => generated 311 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:48:03.304613 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31290f60-af06-4562-8a73-bb4e1661643c None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:48:03.410921 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31290f60-af06-4562-8a73-bb4e1661643c None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:48:03.421487 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:48:03.455297 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd4f5e8-bf46-4b59-ad16-f2dc3b091cd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:48:03.455902 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd4f5e8-bf46-4b59-ad16-f2dc3b091cd3 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:48:03.456178 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd4f5e8-bf46-4b59-ad16-f2dc3b091cd3 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:48:03.456448 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd4f5e8-bf46-4b59-ad16-f2dc3b091cd3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:48:03.457180 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd4f5e8-bf46-4b59-ad16-f2dc3b091cd3 None admin] RBAC: Adding query filter params (role.id=f265386950a641ac8413b53eda4ae34d, scope.project.id=0082e220ecce4c6ab75b86249985f4dd, user.id=edf7c685b37e429595b393c633fed567) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:48:03.457521 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd4f5e8-bf46-4b59-ad16-f2dc3b091cd3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:48:03.458974 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd4f5e8-bf46-4b59-ad16-f2dc3b091cd3 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:48:03.464984 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 573/1146] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Aug 30 15:48:03 2022] GET /identity/v3/role_assignments?user.id=edf7c685b37e429595b393c633fed567&scope.project.id=0082e220ecce4c6ab75b86249985f4dd&role.id=f265386950a641ac8413b53eda4ae34d => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 15:48:05.320839 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba59c3c7-2c55-4761-8388-14fe0a026d88 None None] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:48:05.321439 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba59c3c7-2c55-4761-8388-14fe0a026d88 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:48:05.321439 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba59c3c7-2c55-4761-8388-14fe0a026d88 None None] PATH_INFO: `/` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:48:05.322772 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 574/1147] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 30 15:48:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 15:48:05.329124 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebe6b8f-1fb2-486c-b6e0-7f01ab8b3f45 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:48:05.329904 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebe6b8f-1fb2-486c-b6e0-7f01ab8b3f45 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:48:05.330103 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebe6b8f-1fb2-486c-b6e0-7f01ab8b3f45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:48:05.352917 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-4ebe6b8f-1fb2-486c-b6e0-7f01ab8b3f45 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:48:05.354195 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-4ebe6b8f-1fb2-486c-b6e0-7f01ab8b3f45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:48:05.428050 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 574/1148] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 30 15:48:05 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 15:48:05.692936 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba59c3c7-2c55-4761-8388-14fe0a026d88 None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:48:05.714231 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:48:05.802123 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba59c3c7-2c55-4761-8388-14fe0a026d88 None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:48:05.838134 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9218eb-b1f7-4f99-b6ce-e043b6c06ac6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:48:05.839075 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9218eb-b1f7-4f99-b6ce-e043b6c06ac6 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:48:05.839521 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9218eb-b1f7-4f99-b6ce-e043b6c06ac6 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:48:05.839943 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9218eb-b1f7-4f99-b6ce-e043b6c06ac6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:48:05.857566 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9218eb-b1f7-4f99-b6ce-e043b6c06ac6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 15:48:05.862336 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application [None req-8a9218eb-b1f7-4f99-b6ce-e043b6c06ac6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 15:48:05.862336 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:48:05.862336 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:48:05.862336 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:48:05.862336 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:48:05.862336 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:48:05.862336 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:48:05.862336 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:48:05.862336 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:48:05.862336 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:48:05.862336 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:48:05.862336 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:48:05.862336 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 15:48:05.862336 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 15:48:05.862336 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 15:48:05.862336 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 15:48:05.863451 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 15:48:05.863451 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 15:48:05.863451 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 15:48:05.863451 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 15:48:05.863451 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:48:05.863451 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:48:05.863451 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:48:05.863451 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:48:05.863451 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:48:05.863451 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:48:05.863451 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:48:05.863451 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application with Lock( Aug 30 15:48:05.863451 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:48:05.863451 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:48:05.863451 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:48:05.863451 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:48:05.864679 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:48:05.864679 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:48:05.864679 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:48:05.864679 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:48:05.864679 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 15:48:05.864679 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 15:48:05.864679 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 15:48:05.864679 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 15:48:05.864679 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 15:48:05.864679 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 15:48:05.864679 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 15:48:05.864679 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Aug 30 15:48:05.864679 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 575/1149] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Aug 30 15:48:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 15:48:05.870161 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebe6b8f-1fb2-486c-b6e0-7f01ab8b3f45 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:48:05.974789 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebe6b8f-1fb2-486c-b6e0-7f01ab8b3f45 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:48:05.977025 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:48:06.010371 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b749900-3642-40ea-b31c-11ec98b6beed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:48:06.010968 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b749900-3642-40ea-b31c-11ec98b6beed None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:48:06.011240 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b749900-3642-40ea-b31c-11ec98b6beed None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:48:06.011506 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b749900-3642-40ea-b31c-11ec98b6beed None admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:48:06.012165 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b749900-3642-40ea-b31c-11ec98b6beed None admin] RBAC: Adding query filter params (name=admin) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:48:06.012487 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b749900-3642-40ea-b31c-11ec98b6beed None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:48:06.013890 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b749900-3642-40ea-b31c-11ec98b6beed None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:48:06.022132 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 575/1150] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Aug 30 15:48:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:48:06.028980 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9218eb-b1f7-4f99-b6ce-e043b6c06ac6 None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:48:06.125092 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:48:06.152179 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9218eb-b1f7-4f99-b6ce-e043b6c06ac6 None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:48:06.189479 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e823404-b7c5-4756-8896-2b90f72000f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:48:06.190082 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e823404-b7c5-4756-8896-2b90f72000f2 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:48:06.190355 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e823404-b7c5-4756-8896-2b90f72000f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:48:06.190617 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e823404-b7c5-4756-8896-2b90f72000f2 None admin] PATH_INFO: `/v3/users/edf7c685b37e429595b393c633fed567` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:48:06.191769 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e823404-b7c5-4756-8896-2b90f72000f2 None admin] RBAC: Authorizing `identity:get_user(user_id=edf7c685b37e429595b393c633fed567)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:48:06.193192 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e823404-b7c5-4756-8896-2b90f72000f2 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:48:06.194786 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 576/1151] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Aug 30 15:48:06 2022] GET /identity/v3/users/edf7c685b37e429595b393c633fed567 => generated 286 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:48:06.203534 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b749900-3642-40ea-b31c-11ec98b6beed None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:48:06.310469 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b749900-3642-40ea-b31c-11ec98b6beed None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:48:06.353364 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc31b7a8-f8e5-4197-bf8b-ac3aee323de3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:48:06.354763 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc31b7a8-f8e5-4197-bf8b-ac3aee323de3 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:48:06.355222 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc31b7a8-f8e5-4197-bf8b-ac3aee323de3 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:48:06.355666 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc31b7a8-f8e5-4197-bf8b-ac3aee323de3 None admin] PATH_INFO: `/v3/projects/0082e220ecce4c6ab75b86249985f4dd` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:48:06.357261 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc31b7a8-f8e5-4197-bf8b-ac3aee323de3 None admin] RBAC: Authorizing `identity:get_project(project_id=0082e220ecce4c6ab75b86249985f4dd)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:48:06.358717 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc31b7a8-f8e5-4197-bf8b-ac3aee323de3 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:48:06.360348 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 576/1152] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Aug 30 15:48:06 2022] GET /identity/v3/projects/0082e220ecce4c6ab75b86249985f4dd => generated 311 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:48:06.366430 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e823404-b7c5-4756-8896-2b90f72000f2 None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:48:06.501681 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e823404-b7c5-4756-8896-2b90f72000f2 None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:48:06.539536 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae906ca-4822-47d7-b721-9e2e984d629a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:48:06.540186 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae906ca-4822-47d7-b721-9e2e984d629a None admin] REQUEST_METHOD: `PUT` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:48:06.540463 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae906ca-4822-47d7-b721-9e2e984d629a None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:48:06.540737 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae906ca-4822-47d7-b721-9e2e984d629a None admin] PATH_INFO: `/v3/projects/0082e220ecce4c6ab75b86249985f4dd/users/edf7c685b37e429595b393c633fed567/roles/f265386950a641ac8413b53eda4ae34d` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:48:06.542936 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae906ca-4822-47d7-b721-9e2e984d629a None admin] RBAC: Authorizing `identity:create_grant(project_id=0082e220ecce4c6ab75b86249985f4dd, user_id=edf7c685b37e429595b393c633fed567, role_id=f265386950a641ac8413b53eda4ae34d)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:48:06.544423 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae906ca-4822-47d7-b721-9e2e984d629a None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:48:06.559875 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 577/1153] 10.222.0.20 () {58 vars in 1439 bytes} [Tue Aug 30 15:48:06 2022] PUT /identity/v3/projects/0082e220ecce4c6ab75b86249985f4dd/users/edf7c685b37e429595b393c633fed567/roles/f265386950a641ac8413b53eda4ae34d => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:48:08.433744 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622a6a91-0b9d-46c0-9a9e-7c98e6a0ca13 None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:48:08.434431 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622a6a91-0b9d-46c0-9a9e-7c98e6a0ca13 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:48:08.434431 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622a6a91-0b9d-46c0-9a9e-7c98e6a0ca13 None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:48:08.435652 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 577/1154] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 30 15:48:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 15:48:08.443548 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9a320f-bb4b-4ef8-830e-d3587df38eed None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:48:08.444004 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9a320f-bb4b-4ef8-830e-d3587df38eed None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:48:08.444094 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9a320f-bb4b-4ef8-830e-d3587df38eed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:48:08.453091 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:48:08.465927 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-bd9a320f-bb4b-4ef8-830e-d3587df38eed None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:48:08.467541 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-bd9a320f-bb4b-4ef8-830e-d3587df38eed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:48:08.540387 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 578/1155] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 30 15:48:08 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 15:48:08.806410 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622a6a91-0b9d-46c0-9a9e-7c98e6a0ca13 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:48:08.853047 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:48:08.934686 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622a6a91-0b9d-46c0-9a9e-7c98e6a0ca13 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:48:08.974611 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839d3856-fe8b-481a-9d32-377560fdee11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:48:08.976143 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839d3856-fe8b-481a-9d32-377560fdee11 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:48:08.976854 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839d3856-fe8b-481a-9d32-377560fdee11 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:48:08.977578 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839d3856-fe8b-481a-9d32-377560fdee11 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:48:08.997233 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839d3856-fe8b-481a-9d32-377560fdee11 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 15:48:09.002166 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-839d3856-fe8b-481a-9d32-377560fdee11 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 15:48:09.002166 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:48:09.002166 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:48:09.002166 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:48:09.002166 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:48:09.002166 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:48:09.002166 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:48:09.002166 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:48:09.002166 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:48:09.002166 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:48:09.002166 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:48:09.002166 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:48:09.002166 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 15:48:09.002166 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 15:48:09.002166 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 15:48:09.002166 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 15:48:09.003292 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 15:48:09.003292 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 15:48:09.003292 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 15:48:09.003292 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 15:48:09.003292 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:48:09.003292 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:48:09.003292 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:48:09.003292 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:48:09.003292 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:48:09.003292 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:48:09.003292 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:48:09.003292 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Aug 30 15:48:09.003292 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:48:09.003292 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:48:09.003292 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:48:09.003292 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:48:09.004157 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:48:09.004157 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:48:09.004157 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:48:09.004157 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:48:09.004157 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 15:48:09.004157 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 15:48:09.004157 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 15:48:09.004157 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 15:48:09.004157 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 15:48:09.004157 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 15:48:09.004157 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 15:48:09.004157 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Aug 30 15:48:09.004157 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 578/1156] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Aug 30 15:48:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 15:48:09.009758 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9a320f-bb4b-4ef8-830e-d3587df38eed None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:48:09.112928 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9a320f-bb4b-4ef8-830e-d3587df38eed None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:48:09.148780 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35413996-822e-44d6-8b37-6648e38f5c73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:48:09.149407 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35413996-822e-44d6-8b37-6648e38f5c73 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:48:09.149667 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35413996-822e-44d6-8b37-6648e38f5c73 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:48:09.149940 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35413996-822e-44d6-8b37-6648e38f5c73 None admin] PATH_INFO: `/v3/roles` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:48:09.150599 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35413996-822e-44d6-8b37-6648e38f5c73 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81512) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:48:09.150898 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35413996-822e-44d6-8b37-6648e38f5c73 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:48:09.152270 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35413996-822e-44d6-8b37-6648e38f5c73 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:48:09.153770 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:48:09.165566 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 579/1157] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Aug 30 15:48:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:48:09.175099 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839d3856-fe8b-481a-9d32-377560fdee11 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:48:09.284640 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:48:09.293808 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839d3856-fe8b-481a-9d32-377560fdee11 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:48:09.328883 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c397c526-04e2-4ee9-80e3-a887ff8a6e9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:48:09.329529 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c397c526-04e2-4ee9-80e3-a887ff8a6e9f None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:48:09.329801 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c397c526-04e2-4ee9-80e3-a887ff8a6e9f None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:48:09.330066 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c397c526-04e2-4ee9-80e3-a887ff8a6e9f None admin] PATH_INFO: `/v3/users/edf7c685b37e429595b393c633fed567` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:48:09.331303 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c397c526-04e2-4ee9-80e3-a887ff8a6e9f None admin] RBAC: Authorizing `identity:get_user(user_id=edf7c685b37e429595b393c633fed567)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:48:09.332747 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c397c526-04e2-4ee9-80e3-a887ff8a6e9f None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:48:09.334323 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 579/1158] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Aug 30 15:48:09 2022] GET /identity/v3/users/edf7c685b37e429595b393c633fed567 => generated 286 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:48:09.340578 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35413996-822e-44d6-8b37-6648e38f5c73 None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:48:09.444749 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35413996-822e-44d6-8b37-6648e38f5c73 None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:48:09.480448 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2229627e-6c7b-4d87-84a5-a84587cdf9a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:48:09.481119 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2229627e-6c7b-4d87-84a5-a84587cdf9a2 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:48:09.481438 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2229627e-6c7b-4d87-84a5-a84587cdf9a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:48:09.481704 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2229627e-6c7b-4d87-84a5-a84587cdf9a2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:48:09.555631 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:48:09.563246 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2229627e-6c7b-4d87-84a5-a84587cdf9a2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:48:09.564689 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2229627e-6c7b-4d87-84a5-a84587cdf9a2 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:48:09.665019 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 580/1159] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Aug 30 15:48:09 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 30 15:48:09.672153 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c397c526-04e2-4ee9-80e3-a887ff8a6e9f None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:48:09.785774 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c397c526-04e2-4ee9-80e3-a887ff8a6e9f None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:48:09.820695 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d02210-a2c2-4a85-8b2b-9914ab3acbeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:48:09.821347 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d02210-a2c2-4a85-8b2b-9914ab3acbeb None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:48:09.821626 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d02210-a2c2-4a85-8b2b-9914ab3acbeb None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:48:09.821897 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d02210-a2c2-4a85-8b2b-9914ab3acbeb None admin] PATH_INFO: `/v3/projects/0082e220ecce4c6ab75b86249985f4dd` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:48:09.823145 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d02210-a2c2-4a85-8b2b-9914ab3acbeb None admin] RBAC: Authorizing `identity:get_project(project_id=0082e220ecce4c6ab75b86249985f4dd)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:48:09.824590 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d02210-a2c2-4a85-8b2b-9914ab3acbeb None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:48:09.826271 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 580/1160] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Aug 30 15:48:09 2022] GET /identity/v3/projects/0082e220ecce4c6ab75b86249985f4dd => generated 311 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:48:09.836182 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2229627e-6c7b-4d87-84a5-a84587cdf9a2 None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:48:09.933637 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:48:09.940257 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2229627e-6c7b-4d87-84a5-a84587cdf9a2 None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:48:09.975955 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85de4e37-e8b3-4d97-addd-e6778c3a2b5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:48:09.976556 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85de4e37-e8b3-4d97-addd-e6778c3a2b5f None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:48:09.976835 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85de4e37-e8b3-4d97-addd-e6778c3a2b5f None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:48:09.977105 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85de4e37-e8b3-4d97-addd-e6778c3a2b5f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:48:09.977857 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85de4e37-e8b3-4d97-addd-e6778c3a2b5f None admin] RBAC: Adding query filter params (role.id=f265386950a641ac8413b53eda4ae34d, scope.project.id=0082e220ecce4c6ab75b86249985f4dd, user.id=edf7c685b37e429595b393c633fed567) {{(pid=81512) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:48:09.978167 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85de4e37-e8b3-4d97-addd-e6778c3a2b5f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:48:09.979615 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85de4e37-e8b3-4d97-addd-e6778c3a2b5f None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:48:09.985662 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 581/1161] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Aug 30 15:48:09 2022] GET /identity/v3/role_assignments?user.id=edf7c685b37e429595b393c633fed567&scope.project.id=0082e220ecce4c6ab75b86249985f4dd&role.id=f265386950a641ac8413b53eda4ae34d => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:48:11.931752 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6a7ca9-d409-4fe6-9baf-ee58fb9f7718 None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:48:11.932281 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6a7ca9-d409-4fe6-9baf-ee58fb9f7718 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:48:11.932281 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6a7ca9-d409-4fe6-9baf-ee58fb9f7718 None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:48:11.933588 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 581/1162] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 30 15:48:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 15:48:11.939825 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ce0679-90f1-4e76-a0de-119c121b45ac None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:48:11.940257 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ce0679-90f1-4e76-a0de-119c121b45ac None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:48:11.940367 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ce0679-90f1-4e76-a0de-119c121b45ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:48:11.968806 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-e1ce0679-90f1-4e76-a0de-119c121b45ac None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:48:11.970022 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-e1ce0679-90f1-4e76-a0de-119c121b45ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:48:12.046622 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 582/1163] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 30 15:48:11 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 15:48:12.318404 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6a7ca9-d409-4fe6-9baf-ee58fb9f7718 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:48:12.358213 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:48:12.434583 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6a7ca9-d409-4fe6-9baf-ee58fb9f7718 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:48:12.476418 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2235c637-e760-4ef6-bd1b-19f9c2e1b2f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:48:12.477020 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2235c637-e760-4ef6-bd1b-19f9c2e1b2f2 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:48:12.477329 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2235c637-e760-4ef6-bd1b-19f9c2e1b2f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:48:12.477596 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2235c637-e760-4ef6-bd1b-19f9c2e1b2f2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:48:12.552323 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2235c637-e760-4ef6-bd1b-19f9c2e1b2f2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:48:12.553816 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2235c637-e760-4ef6-bd1b-19f9c2e1b2f2 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:48:12.586167 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:48:12.660695 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 582/1164] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Aug 30 15:48:12 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 30 15:48:12.666874 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ce0679-90f1-4e76-a0de-119c121b45ac None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:48:12.759040 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:48:12.774925 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ce0679-90f1-4e76-a0de-119c121b45ac None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:48:12.812231 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15566b05-baf3-4a1c-8842-bf328ac346a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:48:12.812839 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15566b05-baf3-4a1c-8842-bf328ac346a5 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:48:12.813109 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15566b05-baf3-4a1c-8842-bf328ac346a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:48:12.813408 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15566b05-baf3-4a1c-8842-bf328ac346a5 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:48:12.820296 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15566b05-baf3-4a1c-8842-bf328ac346a5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:48:12.821780 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15566b05-baf3-4a1c-8842-bf328ac346a5 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:48:12.833457 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application [None req-15566b05-baf3-4a1c-8842-bf328ac346a5 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 30 15:48:12.833457 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:48:12.833457 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Aug 30 15:48:12.833457 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Aug 30 15:48:12.833457 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 15:48:12.833457 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 15:48:12.833457 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 15:48:12.833457 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 15:48:12.833457 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Aug 30 15:48:12.833457 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Aug 30 15:48:12.833457 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 15:48:12.833457 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Aug 30 15:48:12.833457 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:48:12.833457 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:48:12.833457 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:48:12.833457 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:48:12.833457 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:48:12.834758 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:48:12.834758 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:48:12.834758 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:48:12.834758 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:48:12.834758 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:48:12.834758 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:48:12.834758 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 15:48:12.834758 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 15:48:12.834758 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 15:48:12.834758 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 15:48:12.834758 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:48:12.834758 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:48:12.834758 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:48:12.834758 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:48:12.834758 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:48:12.834758 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:48:12.835789 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:48:12.835789 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application with Lock( Aug 30 15:48:12.835789 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:48:12.835789 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:48:12.835789 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:48:12.835789 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:48:12.835789 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:48:12.835789 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:48:12.835789 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:48:12.835789 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:48:12.835789 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Aug 30 15:48:12.835789 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 15:48:12.835789 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 30 15:48:12.835789 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Aug 30 15:48:12.835789 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 583/1165] 10.222.0.20 () {56 vars in 1117 bytes} [Tue Aug 30 15:48:12 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 15:48:12.841027 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2235c637-e760-4ef6-bd1b-19f9c2e1b2f2 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:48:12.944223 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2235c637-e760-4ef6-bd1b-19f9c2e1b2f2 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:48:12.973170 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:48:12.980191 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e66e34-34ca-4c9d-91fe-28c57a8e7d52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:48:12.980802 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e66e34-34ca-4c9d-91fe-28c57a8e7d52 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:48:12.981102 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e66e34-34ca-4c9d-91fe-28c57a8e7d52 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:48:12.981391 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e66e34-34ca-4c9d-91fe-28c57a8e7d52 None admin] PATH_INFO: `/v3/domains` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:48:12.982028 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e66e34-34ca-4c9d-91fe-28c57a8e7d52 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:48:12.982328 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e66e34-34ca-4c9d-91fe-28c57a8e7d52 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:48:12.983694 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e66e34-34ca-4c9d-91fe-28c57a8e7d52 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:48:12.990815 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 583/1166] 10.222.0.20 () {56 vars in 1115 bytes} [Tue Aug 30 15:48:12 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:48:14.846337 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658ccc0e-3185-44c7-90d2-760f3f346af3 None None] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:48:14.846893 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658ccc0e-3185-44c7-90d2-760f3f346af3 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:48:14.846893 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658ccc0e-3185-44c7-90d2-760f3f346af3 None None] PATH_INFO: `/` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:48:14.848224 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 584/1167] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 30 15:48:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 15:48:14.854675 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f65203-3310-4b0c-bbc4-e43b57363073 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:48:14.855382 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f65203-3310-4b0c-bbc4-e43b57363073 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:48:14.855531 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f65203-3310-4b0c-bbc4-e43b57363073 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:48:14.876324 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-a0f65203-3310-4b0c-bbc4-e43b57363073 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:48:14.877560 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-a0f65203-3310-4b0c-bbc4-e43b57363073 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:48:14.947602 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 584/1168] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 30 15:48:14 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 15:48:15.222764 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658ccc0e-3185-44c7-90d2-760f3f346af3 None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:48:15.350621 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658ccc0e-3185-44c7-90d2-760f3f346af3 None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:48:15.385212 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d514e875-6f91-40b7-a2d0-e91f81c9e04d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:48:15.385828 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d514e875-6f91-40b7-a2d0-e91f81c9e04d None admin] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:48:15.386099 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d514e875-6f91-40b7-a2d0-e91f81c9e04d None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:48:15.386361 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d514e875-6f91-40b7-a2d0-e91f81c9e04d None admin] PATH_INFO: `/v3/domains` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:48:15.387031 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d514e875-6f91-40b7-a2d0-e91f81c9e04d None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:48:15.388568 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d514e875-6f91-40b7-a2d0-e91f81c9e04d None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:48:15.407792 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 585/1169] 10.222.0.20 () {60 vars in 1138 bytes} [Tue Aug 30 15:48:15 2022] POST /identity/v3/domains => generated 267 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 15:48:17.304996 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7ea309-c567-43e2-9645-da084620baf0 None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:48:17.305927 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7ea309-c567-43e2-9645-da084620baf0 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:48:17.305927 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7ea309-c567-43e2-9645-da084620baf0 None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:48:17.308097 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 585/1170] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 30 15:48:17 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 15:48:17.314903 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa02bad-5367-4b9a-9e5e-282026ecbd2b None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:48:17.315322 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa02bad-5367-4b9a-9e5e-282026ecbd2b None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:48:17.315482 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa02bad-5367-4b9a-9e5e-282026ecbd2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:48:17.344571 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-7aa02bad-5367-4b9a-9e5e-282026ecbd2b None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:48:17.346454 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-7aa02bad-5367-4b9a-9e5e-282026ecbd2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:48:17.350856 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:48:17.426928 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 586/1171] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 30 15:48:17 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 15:48:17.692008 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7ea309-c567-43e2-9645-da084620baf0 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:48:17.802218 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7ea309-c567-43e2-9645-da084620baf0 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:48:17.825643 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:48:17.838250 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c5317a-08f1-44dc-b33c-cd8e90daa8ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:48:17.838875 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c5317a-08f1-44dc-b33c-cd8e90daa8ae None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:48:17.839154 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c5317a-08f1-44dc-b33c-cd8e90daa8ae None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:48:17.839422 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c5317a-08f1-44dc-b33c-cd8e90daa8ae None admin] PATH_INFO: `/v3/domains/a0359e52b90b4e6cafe390480c35545d` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:48:17.851090 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c5317a-08f1-44dc-b33c-cd8e90daa8ae None admin] RBAC: Authorizing `identity:get_domain(domain_id=a0359e52b90b4e6cafe390480c35545d)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:48:17.852546 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c5317a-08f1-44dc-b33c-cd8e90daa8ae None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:48:17.854149 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 586/1172] 10.222.0.20 () {56 vars in 1183 bytes} [Tue Aug 30 15:48:17 2022] GET /identity/v3/domains/a0359e52b90b4e6cafe390480c35545d => generated 267 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:48:17.860986 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa02bad-5367-4b9a-9e5e-282026ecbd2b None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:48:17.967049 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa02bad-5367-4b9a-9e5e-282026ecbd2b None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:48:18.001242 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84034c32-a52c-4447-887e-6c73a8e56bc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:48:18.001861 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84034c32-a52c-4447-887e-6c73a8e56bc3 None admin] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:48:18.002136 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84034c32-a52c-4447-887e-6c73a8e56bc3 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:48:18.002403 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84034c32-a52c-4447-887e-6c73a8e56bc3 None admin] PATH_INFO: `/v3/projects` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:48:18.003050 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84034c32-a52c-4447-887e-6c73a8e56bc3 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:48:18.004492 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84034c32-a52c-4447-887e-6c73a8e56bc3 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:48:18.015671 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:48:18.036075 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 587/1173] 10.222.0.20 () {60 vars in 1141 bytes} [Tue Aug 30 15:48:17 2022] POST /identity/v3/projects => generated 361 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 15:48:19.953740 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7354a6fc-ed8f-46c3-bee5-a4706363c969 None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:48:19.954570 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7354a6fc-ed8f-46c3-bee5-a4706363c969 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:48:19.954570 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7354a6fc-ed8f-46c3-bee5-a4706363c969 None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:48:19.956587 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 587/1174] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 30 15:48:19 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 15:48:19.965266 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa71300-d051-4130-8b5b-d72e3be58f74 None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:48:19.965691 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa71300-d051-4130-8b5b-d72e3be58f74 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:48:19.965830 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa71300-d051-4130-8b5b-d72e3be58f74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:48:19.991436 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-efa71300-d051-4130-8b5b-d72e3be58f74 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:48:19.992959 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-efa71300-d051-4130-8b5b-d72e3be58f74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:48:20.074123 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 588/1175] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 30 15:48:19 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 15:48:20.355222 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7354a6fc-ed8f-46c3-bee5-a4706363c969 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:48:20.472102 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7354a6fc-ed8f-46c3-bee5-a4706363c969 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:48:20.512593 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1792d4aa-1fdf-43de-b61c-f45f525bd5e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:48:20.513249 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1792d4aa-1fdf-43de-b61c-f45f525bd5e3 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:48:20.513534 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1792d4aa-1fdf-43de-b61c-f45f525bd5e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:48:20.513805 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1792d4aa-1fdf-43de-b61c-f45f525bd5e3 None admin] PATH_INFO: `/v3/domains/a0359e52b90b4e6cafe390480c35545d` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:48:20.514978 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1792d4aa-1fdf-43de-b61c-f45f525bd5e3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=a0359e52b90b4e6cafe390480c35545d)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:48:20.516420 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1792d4aa-1fdf-43de-b61c-f45f525bd5e3 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:48:20.518050 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 588/1176] 10.222.0.20 () {56 vars in 1183 bytes} [Tue Aug 30 15:48:20 2022] GET /identity/v3/domains/a0359e52b90b4e6cafe390480c35545d => generated 267 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:48:20.524145 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa71300-d051-4130-8b5b-d72e3be58f74 None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:48:20.642796 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa71300-d051-4130-8b5b-d72e3be58f74 None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:48:20.667688 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:48:20.682795 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cddbfb5-630a-4318-81e3-599daab35b72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:48:20.683405 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cddbfb5-630a-4318-81e3-599daab35b72 None admin] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:48:20.683678 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cddbfb5-630a-4318-81e3-599daab35b72 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:48:20.683946 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cddbfb5-630a-4318-81e3-599daab35b72 None admin] PATH_INFO: `/v3/users` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:48:20.684593 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cddbfb5-630a-4318-81e3-599daab35b72 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:48:20.686068 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cddbfb5-630a-4318-81e3-599daab35b72 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:48:20.710728 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 589/1177] 10.222.0.20 () {60 vars in 1132 bytes} [Tue Aug 30 15:48:20 2022] POST /identity/v3/users => generated 311 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 15:48:22.647261 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbed8af8-6c2d-4dc8-9881-9a8be320f95a None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:48:22.647711 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbed8af8-6c2d-4dc8-9881-9a8be320f95a None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:48:22.647815 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbed8af8-6c2d-4dc8-9881-9a8be320f95a None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:48:22.649954 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 589/1178] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 30 15:48:22 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 15:48:22.655428 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbb13db-9515-4e64-8c94-753e08233127 None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:48:22.655864 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbb13db-9515-4e64-8c94-753e08233127 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:48:22.655966 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbb13db-9515-4e64-8c94-753e08233127 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:48:22.674517 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-7dbb13db-9515-4e64-8c94-753e08233127 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:48:22.675697 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-7dbb13db-9515-4e64-8c94-753e08233127 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:48:22.745580 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 590/1179] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 30 15:48:22 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 15:48:23.011885 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbed8af8-6c2d-4dc8-9881-9a8be320f95a None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:48:23.078795 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:48:23.128642 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbed8af8-6c2d-4dc8-9881-9a8be320f95a None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:48:23.163324 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f86d11-3451-42f5-ae1a-b2b305cea9d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:48:23.163953 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f86d11-3451-42f5-ae1a-b2b305cea9d3 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:48:23.164228 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f86d11-3451-42f5-ae1a-b2b305cea9d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:48:23.164495 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f86d11-3451-42f5-ae1a-b2b305cea9d3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:48:23.175618 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f86d11-3451-42f5-ae1a-b2b305cea9d3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 15:48:23.180416 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-62f86d11-3451-42f5-ae1a-b2b305cea9d3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 15:48:23.180416 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:48:23.180416 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:48:23.180416 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:48:23.180416 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:48:23.180416 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:48:23.180416 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:48:23.180416 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:48:23.180416 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:48:23.180416 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:48:23.180416 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:48:23.180416 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:48:23.180416 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 15:48:23.180416 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 15:48:23.180416 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 15:48:23.180416 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 15:48:23.181627 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 15:48:23.181627 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 15:48:23.181627 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 15:48:23.181627 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 15:48:23.181627 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:48:23.181627 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:48:23.181627 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:48:23.181627 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:48:23.181627 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:48:23.181627 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:48:23.181627 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:48:23.181627 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Aug 30 15:48:23.181627 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:48:23.181627 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:48:23.181627 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:48:23.181627 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:48:23.183324 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:48:23.183324 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:48:23.183324 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:48:23.183324 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:48:23.183324 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 15:48:23.183324 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 15:48:23.183324 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 15:48:23.183324 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 15:48:23.183324 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 15:48:23.183324 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 15:48:23.183324 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 15:48:23.183324 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Aug 30 15:48:23.183324 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 590/1180] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Aug 30 15:48:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 15:48:23.189599 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbb13db-9515-4e64-8c94-753e08233127 None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:48:23.300453 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbb13db-9515-4e64-8c94-753e08233127 None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:48:23.309604 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:48:23.348982 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14e53f0-6729-4dde-b096-e1829e15df17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:48:23.349603 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14e53f0-6729-4dde-b096-e1829e15df17 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:48:23.349882 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14e53f0-6729-4dde-b096-e1829e15df17 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:48:23.350149 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14e53f0-6729-4dde-b096-e1829e15df17 None admin] PATH_INFO: `/v3/roles` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:48:23.350812 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14e53f0-6729-4dde-b096-e1829e15df17 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81512) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:48:23.351107 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14e53f0-6729-4dde-b096-e1829e15df17 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:48:23.352485 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14e53f0-6729-4dde-b096-e1829e15df17 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:48:23.361181 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 591/1181] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Aug 30 15:48:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:48:23.367320 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f86d11-3451-42f5-ae1a-b2b305cea9d3 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:48:23.472816 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f86d11-3451-42f5-ae1a-b2b305cea9d3 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:48:23.485626 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:48:23.518457 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f950cc9a-f9e0-49b5-8769-1ccd03acf051 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:48:23.519078 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f950cc9a-f9e0-49b5-8769-1ccd03acf051 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:48:23.519356 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f950cc9a-f9e0-49b5-8769-1ccd03acf051 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:48:23.519625 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f950cc9a-f9e0-49b5-8769-1ccd03acf051 None admin] PATH_INFO: `/v3/users/d991f8ad9c8242169187749873901c35` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:48:23.533672 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f950cc9a-f9e0-49b5-8769-1ccd03acf051 None admin] RBAC: Authorizing `identity:get_user(user_id=d991f8ad9c8242169187749873901c35)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:48:23.535125 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f950cc9a-f9e0-49b5-8769-1ccd03acf051 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:48:23.536718 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 591/1182] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Aug 30 15:48:23 2022] GET /identity/v3/users/d991f8ad9c8242169187749873901c35 => generated 311 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:48:23.543307 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14e53f0-6729-4dde-b096-e1829e15df17 None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:48:23.657258 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14e53f0-6729-4dde-b096-e1829e15df17 None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:48:23.698077 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514eaeeb-be02-48a6-9791-d30f5c523f93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:48:23.698677 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514eaeeb-be02-48a6-9791-d30f5c523f93 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:48:23.698959 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514eaeeb-be02-48a6-9791-d30f5c523f93 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:48:23.699210 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514eaeeb-be02-48a6-9791-d30f5c523f93 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:48:23.742141 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:48:23.780399 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514eaeeb-be02-48a6-9791-d30f5c523f93 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:48:23.781881 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514eaeeb-be02-48a6-9791-d30f5c523f93 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:48:23.894773 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 592/1183] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Aug 30 15:48:23 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 30 15:48:23.901942 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f950cc9a-f9e0-49b5-8769-1ccd03acf051 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:48:24.013745 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f950cc9a-f9e0-49b5-8769-1ccd03acf051 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:48:24.050666 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0bdf6d-fba9-4647-a618-5349f8e33c57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:48:24.051302 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0bdf6d-fba9-4647-a618-5349f8e33c57 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:48:24.051578 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0bdf6d-fba9-4647-a618-5349f8e33c57 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:48:24.051839 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0bdf6d-fba9-4647-a618-5349f8e33c57 None admin] PATH_INFO: `/v3/projects/87ded756db5845eeab8a3f5b648fb90c` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:48:24.053057 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0bdf6d-fba9-4647-a618-5349f8e33c57 None admin] RBAC: Authorizing `identity:get_project(project_id=87ded756db5845eeab8a3f5b648fb90c)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:48:24.054519 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0bdf6d-fba9-4647-a618-5349f8e33c57 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:48:24.056135 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 592/1184] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Aug 30 15:48:23 2022] GET /identity/v3/projects/87ded756db5845eeab8a3f5b648fb90c => generated 361 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:48:24.064223 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514eaeeb-be02-48a6-9791-d30f5c523f93 None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:48:24.148374 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:48:24.180517 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514eaeeb-be02-48a6-9791-d30f5c523f93 None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:48:24.219446 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca515df9-3518-425b-b600-35b83af30575 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:48:24.220049 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca515df9-3518-425b-b600-35b83af30575 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:48:24.220326 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca515df9-3518-425b-b600-35b83af30575 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:48:24.220595 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca515df9-3518-425b-b600-35b83af30575 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:48:24.221347 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca515df9-3518-425b-b600-35b83af30575 None admin] RBAC: Adding query filter params (role.id=f265386950a641ac8413b53eda4ae34d, scope.project.id=87ded756db5845eeab8a3f5b648fb90c, user.id=d991f8ad9c8242169187749873901c35) {{(pid=81512) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:48:24.221655 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca515df9-3518-425b-b600-35b83af30575 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:48:24.223092 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca515df9-3518-425b-b600-35b83af30575 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:48:24.228652 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 593/1185] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Aug 30 15:48:24 2022] GET /identity/v3/role_assignments?user.id=d991f8ad9c8242169187749873901c35&scope.project.id=87ded756db5845eeab8a3f5b648fb90c&role.id=f265386950a641ac8413b53eda4ae34d => generated 259 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:48:26.081886 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35529df2-2c6d-444c-a023-d5b85c21c9ec None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:48:26.082728 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35529df2-2c6d-444c-a023-d5b85c21c9ec None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:48:26.082728 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35529df2-2c6d-444c-a023-d5b85c21c9ec None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:48:26.084842 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 593/1186] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 30 15:48:26 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 15:48:26.091131 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb93ad1-2db2-4841-9276-eb4efd65905d None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:48:26.091578 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb93ad1-2db2-4841-9276-eb4efd65905d None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:48:26.091708 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb93ad1-2db2-4841-9276-eb4efd65905d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:48:26.117035 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-efb93ad1-2db2-4841-9276-eb4efd65905d None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:48:26.118587 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-efb93ad1-2db2-4841-9276-eb4efd65905d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:48:26.188357 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 594/1187] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 30 15:48:26 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 15:48:26.455378 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35529df2-2c6d-444c-a023-d5b85c21c9ec None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:48:26.501076 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:48:26.563782 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35529df2-2c6d-444c-a023-d5b85c21c9ec None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:48:26.598920 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598c25eb-c3aa-43a5-8019-33496dc2356a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:48:26.599534 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598c25eb-c3aa-43a5-8019-33496dc2356a None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:48:26.599804 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598c25eb-c3aa-43a5-8019-33496dc2356a None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:48:26.600060 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598c25eb-c3aa-43a5-8019-33496dc2356a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:48:26.610946 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598c25eb-c3aa-43a5-8019-33496dc2356a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 15:48:26.615659 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-598c25eb-c3aa-43a5-8019-33496dc2356a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 15:48:26.615659 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:48:26.615659 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:48:26.615659 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:48:26.615659 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:48:26.615659 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:48:26.615659 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:48:26.615659 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:48:26.615659 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:48:26.615659 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:48:26.615659 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:48:26.615659 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:48:26.615659 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 15:48:26.615659 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 15:48:26.615659 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 15:48:26.615659 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 15:48:26.616716 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 15:48:26.616716 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 15:48:26.616716 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 15:48:26.616716 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 15:48:26.616716 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:48:26.616716 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:48:26.616716 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:48:26.616716 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:48:26.616716 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:48:26.616716 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:48:26.616716 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:48:26.616716 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Aug 30 15:48:26.616716 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:48:26.616716 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:48:26.616716 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:48:26.616716 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:48:26.617721 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:48:26.617721 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:48:26.617721 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:48:26.617721 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:48:26.617721 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 15:48:26.617721 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 15:48:26.617721 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 15:48:26.617721 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 15:48:26.617721 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 15:48:26.617721 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 15:48:26.617721 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 15:48:26.617721 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Aug 30 15:48:26.617721 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 594/1188] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Aug 30 15:48:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 15:48:26.623732 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb93ad1-2db2-4841-9276-eb4efd65905d None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:48:26.696372 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:48:26.736467 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb93ad1-2db2-4841-9276-eb4efd65905d None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:48:26.771753 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846bb54c-b9a5-4732-aff1-d495cd8f02b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:48:26.772350 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846bb54c-b9a5-4732-aff1-d495cd8f02b2 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:48:26.772632 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846bb54c-b9a5-4732-aff1-d495cd8f02b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:48:26.772897 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846bb54c-b9a5-4732-aff1-d495cd8f02b2 None admin] PATH_INFO: `/v3/roles` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:48:26.773576 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846bb54c-b9a5-4732-aff1-d495cd8f02b2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81512) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:48:26.773877 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846bb54c-b9a5-4732-aff1-d495cd8f02b2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:48:26.775244 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846bb54c-b9a5-4732-aff1-d495cd8f02b2 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:48:26.783412 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 595/1189] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Aug 30 15:48:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:48:26.792277 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598c25eb-c3aa-43a5-8019-33496dc2356a None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:48:26.901633 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:48:26.905986 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598c25eb-c3aa-43a5-8019-33496dc2356a None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:48:26.939509 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b803f6e-7cd4-4f95-8f86-89def6e10a88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:48:26.940118 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b803f6e-7cd4-4f95-8f86-89def6e10a88 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:48:26.940389 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b803f6e-7cd4-4f95-8f86-89def6e10a88 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:48:26.940651 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b803f6e-7cd4-4f95-8f86-89def6e10a88 None admin] PATH_INFO: `/v3/users/d991f8ad9c8242169187749873901c35` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:48:26.941984 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b803f6e-7cd4-4f95-8f86-89def6e10a88 None admin] RBAC: Authorizing `identity:get_user(user_id=d991f8ad9c8242169187749873901c35)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:48:26.943409 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b803f6e-7cd4-4f95-8f86-89def6e10a88 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:48:26.944950 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 595/1190] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Aug 30 15:48:26 2022] GET /identity/v3/users/d991f8ad9c8242169187749873901c35 => generated 311 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:48:26.951652 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846bb54c-b9a5-4732-aff1-d495cd8f02b2 None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:48:27.071375 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846bb54c-b9a5-4732-aff1-d495cd8f02b2 None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:48:27.099773 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:48:27.109386 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5155ffa-b4ea-4a26-91a2-b03f98f17823 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:48:27.110005 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5155ffa-b4ea-4a26-91a2-b03f98f17823 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:48:27.110273 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5155ffa-b4ea-4a26-91a2-b03f98f17823 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:48:27.110540 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5155ffa-b4ea-4a26-91a2-b03f98f17823 None admin] PATH_INFO: `/v3/projects/87ded756db5845eeab8a3f5b648fb90c` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:48:27.111668 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5155ffa-b4ea-4a26-91a2-b03f98f17823 None admin] RBAC: Authorizing `identity:get_project(project_id=87ded756db5845eeab8a3f5b648fb90c)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:48:27.113106 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5155ffa-b4ea-4a26-91a2-b03f98f17823 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:48:27.114743 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 596/1191] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Aug 30 15:48:26 2022] GET /identity/v3/projects/87ded756db5845eeab8a3f5b648fb90c => generated 361 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:48:27.124075 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b803f6e-7cd4-4f95-8f86-89def6e10a88 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:48:27.241416 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b803f6e-7cd4-4f95-8f86-89def6e10a88 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:48:27.277806 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc6d7ac-ff83-4f50-a56e-1459055efd83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:48:27.278442 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc6d7ac-ff83-4f50-a56e-1459055efd83 None admin] REQUEST_METHOD: `PUT` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:48:27.278712 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc6d7ac-ff83-4f50-a56e-1459055efd83 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:48:27.278977 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc6d7ac-ff83-4f50-a56e-1459055efd83 None admin] PATH_INFO: `/v3/projects/87ded756db5845eeab8a3f5b648fb90c/users/d991f8ad9c8242169187749873901c35/roles/f265386950a641ac8413b53eda4ae34d` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:48:27.281167 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc6d7ac-ff83-4f50-a56e-1459055efd83 None admin] RBAC: Authorizing `identity:create_grant(project_id=87ded756db5845eeab8a3f5b648fb90c, user_id=d991f8ad9c8242169187749873901c35, role_id=f265386950a641ac8413b53eda4ae34d)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:48:27.282724 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc6d7ac-ff83-4f50-a56e-1459055efd83 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:48:27.298670 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 596/1192] 10.222.0.20 () {58 vars in 1439 bytes} [Tue Aug 30 15:48:27 2022] PUT /identity/v3/projects/87ded756db5845eeab8a3f5b648fb90c/users/d991f8ad9c8242169187749873901c35/roles/f265386950a641ac8413b53eda4ae34d => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:48:29.171554 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ceca8e1-7618-4aca-920c-9106cb4c5313 None None] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:48:29.172127 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ceca8e1-7618-4aca-920c-9106cb4c5313 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:48:29.172127 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ceca8e1-7618-4aca-920c-9106cb4c5313 None None] PATH_INFO: `/` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:48:29.173496 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 597/1193] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 30 15:48:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 15:48:29.179829 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf0645b-84a9-48a9-ad10-8196357628d2 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:48:29.180576 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf0645b-84a9-48a9-ad10-8196357628d2 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:48:29.180705 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf0645b-84a9-48a9-ad10-8196357628d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:48:29.201230 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-bcf0645b-84a9-48a9-ad10-8196357628d2 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:48:29.202441 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-bcf0645b-84a9-48a9-ad10-8196357628d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:48:29.223059 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:48:29.275046 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 597/1194] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 30 15:48:29 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 15:48:29.546319 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ceca8e1-7618-4aca-920c-9106cb4c5313 None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:48:29.675267 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ceca8e1-7618-4aca-920c-9106cb4c5313 None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:48:29.714432 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2c54be-a556-4566-a84d-6222562299aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:48:29.715058 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2c54be-a556-4566-a84d-6222562299aa None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:48:29.715334 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2c54be-a556-4566-a84d-6222562299aa None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:48:29.715632 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2c54be-a556-4566-a84d-6222562299aa None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:48:29.727803 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2c54be-a556-4566-a84d-6222562299aa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 15:48:29.732643 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application [None req-2a2c54be-a556-4566-a84d-6222562299aa None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 15:48:29.732643 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:48:29.732643 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:48:29.732643 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:48:29.732643 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:48:29.732643 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:48:29.732643 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:48:29.732643 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:48:29.732643 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:48:29.732643 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:48:29.732643 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:48:29.732643 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:48:29.732643 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 15:48:29.732643 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 15:48:29.732643 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 15:48:29.732643 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 15:48:29.733775 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 15:48:29.733775 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 15:48:29.733775 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 15:48:29.733775 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 15:48:29.733775 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:48:29.733775 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:48:29.733775 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:48:29.733775 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:48:29.733775 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:48:29.733775 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:48:29.733775 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:48:29.733775 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application with Lock( Aug 30 15:48:29.733775 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:48:29.733775 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:48:29.733775 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:48:29.733775 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:48:29.734643 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:48:29.734643 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:48:29.734643 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:48:29.734643 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:48:29.734643 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 15:48:29.734643 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 15:48:29.734643 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 15:48:29.734643 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 15:48:29.734643 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 15:48:29.734643 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 15:48:29.734643 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 15:48:29.734643 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Aug 30 15:48:29.734643 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 598/1195] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Aug 30 15:48:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 15:48:29.740168 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf0645b-84a9-48a9-ad10-8196357628d2 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:48:29.855189 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf0645b-84a9-48a9-ad10-8196357628d2 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:48:29.891918 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0025f82-befe-406e-ad37-12b1e58abc44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:48:29.892536 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0025f82-befe-406e-ad37-12b1e58abc44 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:48:29.892814 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0025f82-befe-406e-ad37-12b1e58abc44 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:48:29.893086 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0025f82-befe-406e-ad37-12b1e58abc44 None admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:48:29.893777 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0025f82-befe-406e-ad37-12b1e58abc44 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:48:29.894073 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0025f82-befe-406e-ad37-12b1e58abc44 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:48:29.895471 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0025f82-befe-406e-ad37-12b1e58abc44 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:48:29.903609 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 598/1196] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Aug 30 15:48:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:48:29.910624 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2c54be-a556-4566-a84d-6222562299aa None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:48:29.985124 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:48:30.033937 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2c54be-a556-4566-a84d-6222562299aa None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:48:30.070153 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bef8f8-743c-4b7d-8dd5-7308bda4bf0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:48:30.070764 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bef8f8-743c-4b7d-8dd5-7308bda4bf0b None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:48:30.071043 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bef8f8-743c-4b7d-8dd5-7308bda4bf0b None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:48:30.071309 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bef8f8-743c-4b7d-8dd5-7308bda4bf0b None admin] PATH_INFO: `/v3/users/d991f8ad9c8242169187749873901c35` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:48:30.072460 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bef8f8-743c-4b7d-8dd5-7308bda4bf0b None admin] RBAC: Authorizing `identity:get_user(user_id=d991f8ad9c8242169187749873901c35)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:48:30.073935 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bef8f8-743c-4b7d-8dd5-7308bda4bf0b None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:48:30.075528 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 599/1197] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Aug 30 15:48:29 2022] GET /identity/v3/users/d991f8ad9c8242169187749873901c35 => generated 311 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:48:30.081742 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0025f82-befe-406e-ad37-12b1e58abc44 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:48:30.101681 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:48:30.190123 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0025f82-befe-406e-ad37-12b1e58abc44 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:48:30.224752 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779a3785-017b-4b81-8bff-2c4d046f57ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:48:30.225397 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779a3785-017b-4b81-8bff-2c4d046f57ce None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:48:30.225674 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779a3785-017b-4b81-8bff-2c4d046f57ce None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:48:30.225923 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779a3785-017b-4b81-8bff-2c4d046f57ce None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:48:30.295144 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779a3785-017b-4b81-8bff-2c4d046f57ce None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:48:30.296602 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779a3785-017b-4b81-8bff-2c4d046f57ce None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:48:30.314651 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:48:30.401014 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 599/1198] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Aug 30 15:48:30 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 30 15:48:30.410580 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bef8f8-743c-4b7d-8dd5-7308bda4bf0b None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:48:30.523068 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bef8f8-743c-4b7d-8dd5-7308bda4bf0b None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:48:30.546897 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:48:30.560408 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa526783-d0ce-414f-9fbc-e7b126e9889e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:48:30.561022 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa526783-d0ce-414f-9fbc-e7b126e9889e None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:48:30.561325 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa526783-d0ce-414f-9fbc-e7b126e9889e None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:48:30.561590 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa526783-d0ce-414f-9fbc-e7b126e9889e None admin] PATH_INFO: `/v3/projects/87ded756db5845eeab8a3f5b648fb90c` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:48:30.562737 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa526783-d0ce-414f-9fbc-e7b126e9889e None admin] RBAC: Authorizing `identity:get_project(project_id=87ded756db5845eeab8a3f5b648fb90c)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:48:30.564171 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa526783-d0ce-414f-9fbc-e7b126e9889e None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:48:30.565821 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 600/1199] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Aug 30 15:48:30 2022] GET /identity/v3/projects/87ded756db5845eeab8a3f5b648fb90c => generated 361 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:48:30.572197 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779a3785-017b-4b81-8bff-2c4d046f57ce None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:48:30.677895 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779a3785-017b-4b81-8bff-2c4d046f57ce None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:48:30.696274 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:48:30.722472 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d253a22f-c705-40d0-8cc2-3563326e358f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:48:30.723068 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d253a22f-c705-40d0-8cc2-3563326e358f None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:48:30.723339 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d253a22f-c705-40d0-8cc2-3563326e358f None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:48:30.723607 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d253a22f-c705-40d0-8cc2-3563326e358f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:48:30.724327 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d253a22f-c705-40d0-8cc2-3563326e358f None admin] RBAC: Adding query filter params (role.id=f265386950a641ac8413b53eda4ae34d, scope.project.id=87ded756db5845eeab8a3f5b648fb90c, user.id=d991f8ad9c8242169187749873901c35) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:48:30.724628 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d253a22f-c705-40d0-8cc2-3563326e358f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:48:30.726178 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d253a22f-c705-40d0-8cc2-3563326e358f None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:48:30.732091 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 600/1200] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Aug 30 15:48:30 2022] GET /identity/v3/role_assignments?user.id=d991f8ad9c8242169187749873901c35&scope.project.id=87ded756db5845eeab8a3f5b648fb90c&role.id=f265386950a641ac8413b53eda4ae34d => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:48:40.555541 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=5 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:48:40.729388 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=9 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:49:26.109875 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080106cf-05bf-4e2b-a544-298a65507925 None None] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:49:26.109875 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080106cf-05bf-4e2b-a544-298a65507925 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:49:26.109875 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080106cf-05bf-4e2b-a544-298a65507925 None None] PATH_INFO: `/` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:49:26.109875 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 601/1201] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 30 15:49:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 15:49:26.116469 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efd2d6d-e57e-428e-8dfd-dd562a20155c None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:49:26.117475 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efd2d6d-e57e-428e-8dfd-dd562a20155c None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:49:26.117965 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efd2d6d-e57e-428e-8dfd-dd562a20155c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:49:26.141866 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-3efd2d6d-e57e-428e-8dfd-dd562a20155c None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:49:26.143399 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-3efd2d6d-e57e-428e-8dfd-dd562a20155c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:49:26.222760 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 601/1202] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 30 15:49:26 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 15:49:26.501319 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080106cf-05bf-4e2b-a544-298a65507925 None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:49:26.622740 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080106cf-05bf-4e2b-a544-298a65507925 None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:49:26.679386 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4159eea7-efaa-4687-a218-d252f6c242a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:49:26.680342 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4159eea7-efaa-4687-a218-d252f6c242a8 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:49:26.680780 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4159eea7-efaa-4687-a218-d252f6c242a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:49:26.681286 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4159eea7-efaa-4687-a218-d252f6c242a8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:49:26.694463 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4159eea7-efaa-4687-a218-d252f6c242a8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:49:26.695962 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4159eea7-efaa-4687-a218-d252f6c242a8 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:49:26.713117 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application [None req-4159eea7-efaa-4687-a218-d252f6c242a8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 15:49:26.713117 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:49:26.713117 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:49:26.713117 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:49:26.713117 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:49:26.713117 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:49:26.713117 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:49:26.713117 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:49:26.713117 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:49:26.713117 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:49:26.713117 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:49:26.713117 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:49:26.713117 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 15:49:26.713117 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 15:49:26.713117 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 15:49:26.713117 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 15:49:26.714335 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:49:26.714335 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:49:26.714335 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:49:26.714335 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:49:26.714335 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:49:26.714335 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:49:26.714335 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:49:26.714335 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application with Lock( Aug 30 15:49:26.714335 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:49:26.714335 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:49:26.714335 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:49:26.714335 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:49:26.714335 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:49:26.714335 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:49:26.714335 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:49:26.714335 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:49:26.714335 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 15:49:26.715344 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 15:49:26.715344 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 15:49:26.715344 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Aug 30 15:49:26.715344 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 602/1203] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 30 15:49:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 15:49:26.721039 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efd2d6d-e57e-428e-8dfd-dd562a20155c None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:49:26.841982 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efd2d6d-e57e-428e-8dfd-dd562a20155c None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:49:26.881370 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:49:26.885040 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136196c3-747a-4b04-88af-d17f27803eb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:49:26.885711 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136196c3-747a-4b04-88af-d17f27803eb8 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:49:26.885966 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136196c3-747a-4b04-88af-d17f27803eb8 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:49:26.886294 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136196c3-747a-4b04-88af-d17f27803eb8 None admin] PATH_INFO: `/v3/domains` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:49:26.886934 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136196c3-747a-4b04-88af-d17f27803eb8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:49:26.887453 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136196c3-747a-4b04-88af-d17f27803eb8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:49:26.888982 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136196c3-747a-4b04-88af-d17f27803eb8 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:49:26.901337 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 602/1204] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Aug 30 15:49:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:49:26.907362 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4159eea7-efaa-4687-a218-d252f6c242a8 None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:49:26.981144 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:49:27.046221 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4159eea7-efaa-4687-a218-d252f6c242a8 None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:49:27.090819 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba3c289-5b8a-4688-b578-064261ef1fb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:49:27.091592 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba3c289-5b8a-4688-b578-064261ef1fb8 None admin] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:49:27.091875 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba3c289-5b8a-4688-b578-064261ef1fb8 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:49:27.092257 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba3c289-5b8a-4688-b578-064261ef1fb8 None admin] PATH_INFO: `/v3/users` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:49:27.092857 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba3c289-5b8a-4688-b578-064261ef1fb8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:49:27.094350 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba3c289-5b8a-4688-b578-064261ef1fb8 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:49:27.121211 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 603/1205] 10.222.0.20 () {60 vars in 1132 bytes} [Tue Aug 30 15:49:26 2022] POST /identity/v3/users => generated 251 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 15:49:29.263886 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198545f0-03bf-4c88-8603-f349c77961c5 None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:49:29.264486 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198545f0-03bf-4c88-8603-f349c77961c5 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:49:29.264486 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198545f0-03bf-4c88-8603-f349c77961c5 None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:49:29.266559 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 603/1206] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 30 15:49:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 15:49:29.271715 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f50374-9d4c-4576-8860-1cbd771a21e1 None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:49:29.272195 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f50374-9d4c-4576-8860-1cbd771a21e1 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:49:29.272281 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f50374-9d4c-4576-8860-1cbd771a21e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:49:29.291536 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-b1f50374-9d4c-4576-8860-1cbd771a21e1 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:49:29.293046 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-b1f50374-9d4c-4576-8860-1cbd771a21e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:49:29.383335 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 604/1207] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 30 15:49:29 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 15:49:29.654502 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198545f0-03bf-4c88-8603-f349c77961c5 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:49:29.776727 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198545f0-03bf-4c88-8603-f349c77961c5 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:49:29.813631 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5b9fc5-7b2c-4bba-9c13-77c811a5b6d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:49:29.814635 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5b9fc5-7b2c-4bba-9c13-77c811a5b6d6 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:49:29.815119 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5b9fc5-7b2c-4bba-9c13-77c811a5b6d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:49:29.816497 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5b9fc5-7b2c-4bba-9c13-77c811a5b6d6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:49:29.831067 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5b9fc5-7b2c-4bba-9c13-77c811a5b6d6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 15:49:29.836266 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-3a5b9fc5-7b2c-4bba-9c13-77c811a5b6d6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 15:49:29.836266 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:49:29.836266 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:49:29.836266 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:49:29.836266 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:49:29.836266 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:49:29.836266 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:49:29.836266 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:49:29.836266 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:49:29.836266 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:49:29.836266 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:49:29.836266 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:49:29.836266 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 15:49:29.836266 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 15:49:29.836266 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 15:49:29.836266 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 15:49:29.837349 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 15:49:29.837349 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 15:49:29.837349 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 15:49:29.837349 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 15:49:29.837349 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:49:29.837349 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:49:29.837349 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:49:29.837349 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:49:29.837349 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:49:29.837349 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:49:29.837349 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:49:29.837349 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Aug 30 15:49:29.837349 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:49:29.837349 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:49:29.837349 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:49:29.837349 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:49:29.838244 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:49:29.838244 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:49:29.838244 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:49:29.838244 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:49:29.838244 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 15:49:29.838244 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 15:49:29.838244 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 15:49:29.838244 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 15:49:29.838244 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 15:49:29.838244 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 15:49:29.838244 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 15:49:29.838244 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Aug 30 15:49:29.838244 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 604/1208] 10.222.0.20 () {56 vars in 1102 bytes} [Tue Aug 30 15:49:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 15:49:29.843965 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f50374-9d4c-4576-8860-1cbd771a21e1 None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:49:29.888480 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:49:29.952932 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f50374-9d4c-4576-8860-1cbd771a21e1 None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:49:30.003663 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84898eee-66ac-43c8-8df2-9b7d742a9078 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:49:30.004397 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84898eee-66ac-43c8-8df2-9b7d742a9078 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:49:30.004799 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84898eee-66ac-43c8-8df2-9b7d742a9078 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:49:30.005145 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84898eee-66ac-43c8-8df2-9b7d742a9078 None admin] PATH_INFO: `/v3/roles` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:49:30.006167 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84898eee-66ac-43c8-8df2-9b7d742a9078 None admin] RBAC: Adding query filter params (name=service) {{(pid=81512) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:49:30.006599 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84898eee-66ac-43c8-8df2-9b7d742a9078 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:49:30.008569 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84898eee-66ac-43c8-8df2-9b7d742a9078 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:49:30.019461 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 605/1209] 10.222.0.20 () {56 vars in 1103 bytes} [Tue Aug 30 15:49:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:49:30.025675 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5b9fc5-7b2c-4bba-9c13-77c811a5b6d6 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:49:30.083646 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:49:30.145887 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5b9fc5-7b2c-4bba-9c13-77c811a5b6d6 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:49:30.183214 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2584295-1854-4513-a3d0-bb5e767b0630 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:49:30.183958 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2584295-1854-4513-a3d0-bb5e767b0630 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:49:30.184233 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2584295-1854-4513-a3d0-bb5e767b0630 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:49:30.184688 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2584295-1854-4513-a3d0-bb5e767b0630 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:49:30.197298 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2584295-1854-4513-a3d0-bb5e767b0630 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:49:30.198762 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2584295-1854-4513-a3d0-bb5e767b0630 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:49:30.214264 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-a2584295-1854-4513-a3d0-bb5e767b0630 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 15:49:30.214264 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:49:30.214264 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:49:30.214264 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:49:30.214264 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:49:30.214264 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:49:30.214264 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:49:30.214264 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:49:30.214264 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:49:30.214264 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:49:30.214264 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:49:30.214264 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:49:30.214264 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 15:49:30.214264 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 15:49:30.214264 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 15:49:30.214264 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 15:49:30.215624 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:49:30.215624 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:49:30.215624 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:49:30.215624 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:49:30.215624 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:49:30.215624 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:49:30.215624 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:49:30.215624 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Aug 30 15:49:30.215624 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:49:30.215624 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:49:30.215624 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:49:30.215624 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:49:30.215624 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:49:30.215624 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:49:30.215624 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:49:30.215624 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:49:30.215624 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 15:49:30.216747 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 15:49:30.216747 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 15:49:30.216747 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Aug 30 15:49:30.216747 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 605/1210] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 30 15:49:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 15:49:30.223767 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84898eee-66ac-43c8-8df2-9b7d742a9078 None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:49:30.330793 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:49:30.334039 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84898eee-66ac-43c8-8df2-9b7d742a9078 None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:49:30.370737 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a040c7da-40bb-437f-bcc2-7b9a6a8f3479 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:49:30.371351 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a040c7da-40bb-437f-bcc2-7b9a6a8f3479 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:49:30.371629 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a040c7da-40bb-437f-bcc2-7b9a6a8f3479 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:49:30.372014 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a040c7da-40bb-437f-bcc2-7b9a6a8f3479 None admin] PATH_INFO: `/v3/domains` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:49:30.372661 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a040c7da-40bb-437f-bcc2-7b9a6a8f3479 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81512) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:49:30.372949 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a040c7da-40bb-437f-bcc2-7b9a6a8f3479 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:49:30.374414 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a040c7da-40bb-437f-bcc2-7b9a6a8f3479 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:49:30.387256 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 606/1211] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Aug 30 15:49:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:49:30.393083 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2584295-1854-4513-a3d0-bb5e767b0630 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:49:30.505053 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2584295-1854-4513-a3d0-bb5e767b0630 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:49:30.505053 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:49:30.541184 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4359900-fe26-4665-adb8-5688a79943c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:49:30.542014 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4359900-fe26-4665-adb8-5688a79943c7 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:49:30.542298 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4359900-fe26-4665-adb8-5688a79943c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:49:30.542575 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4359900-fe26-4665-adb8-5688a79943c7 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:49:30.551798 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4359900-fe26-4665-adb8-5688a79943c7 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:49:30.553306 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4359900-fe26-4665-adb8-5688a79943c7 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:49:30.565115 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-c4359900-fe26-4665-adb8-5688a79943c7 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 30 15:49:30.565115 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:49:30.565115 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:49:30.565115 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:49:30.565115 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:49:30.565115 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:49:30.565115 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:49:30.565115 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:49:30.565115 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:49:30.565115 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:49:30.565115 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:49:30.565115 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:49:30.565115 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 15:49:30.565115 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 15:49:30.565115 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 15:49:30.565115 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 15:49:30.566638 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:49:30.566638 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:49:30.566638 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 15:49:30.566638 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 15:49:30.566638 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 15:49:30.566638 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 15:49:30.566638 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:49:30.566638 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:49:30.566638 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:49:30.566638 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:49:30.566638 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:49:30.566638 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Aug 30 15:49:30.566638 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:49:30.566638 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:49:30.566638 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:49:30.566638 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:49:30.566638 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:49:30.566638 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:49:30.568085 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:49:30.568085 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:49:30.568085 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 15:49:30.568085 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 15:49:30.568085 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 15:49:30.568085 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 15:49:30.568085 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 15:49:30.568085 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 15:49:30.568085 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 30 15:49:30.568085 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Aug 30 15:49:30.569431 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 606/1212] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 30 15:49:30 2022] GET /identity/v3/users/placement => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 15:49:30.572276 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a040c7da-40bb-437f-bcc2-7b9a6a8f3479 None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:49:30.683172 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a040c7da-40bb-437f-bcc2-7b9a6a8f3479 None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:49:30.722746 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5814f991-e41d-4e1e-862c-b99dcdb5c463 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:49:30.723989 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5814f991-e41d-4e1e-862c-b99dcdb5c463 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:49:30.723989 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5814f991-e41d-4e1e-862c-b99dcdb5c463 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:49:30.724218 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5814f991-e41d-4e1e-862c-b99dcdb5c463 None admin] PATH_INFO: `/v3/users` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:49:30.725000 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5814f991-e41d-4e1e-862c-b99dcdb5c463 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81512) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:49:30.725448 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5814f991-e41d-4e1e-862c-b99dcdb5c463 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:49:30.726960 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5814f991-e41d-4e1e-862c-b99dcdb5c463 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:49:30.746146 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 607/1213] 10.222.0.20 () {56 vars in 1143 bytes} [Tue Aug 30 15:49:30 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:49:30.752389 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4359900-fe26-4665-adb8-5688a79943c7 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:49:30.871225 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4359900-fe26-4665-adb8-5688a79943c7 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:49:30.916402 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5104efd0-7aa8-4430-9ca2-ad12644d9d56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:49:30.917402 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5104efd0-7aa8-4430-9ca2-ad12644d9d56 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:49:30.917685 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5104efd0-7aa8-4430-9ca2-ad12644d9d56 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:49:30.917957 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5104efd0-7aa8-4430-9ca2-ad12644d9d56 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:49:30.952652 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:49:31.003364 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5104efd0-7aa8-4430-9ca2-ad12644d9d56 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:49:31.006085 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5104efd0-7aa8-4430-9ca2-ad12644d9d56 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:49:31.131842 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 607/1214] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Aug 30 15:49:30 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 30 15:49:31.138042 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5814f991-e41d-4e1e-862c-b99dcdb5c463 None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:49:31.158378 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:49:31.249595 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5814f991-e41d-4e1e-862c-b99dcdb5c463 None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:49:31.285342 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4458542c-f644-4a15-9981-c5311387fbc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:49:31.286018 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4458542c-f644-4a15-9981-c5311387fbc4 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:49:31.286306 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4458542c-f644-4a15-9981-c5311387fbc4 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:49:31.286577 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4458542c-f644-4a15-9981-c5311387fbc4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:49:31.297118 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4458542c-f644-4a15-9981-c5311387fbc4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:49:31.298670 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4458542c-f644-4a15-9981-c5311387fbc4 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:49:31.314233 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application [None req-4458542c-f644-4a15-9981-c5311387fbc4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 15:49:31.314233 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:49:31.314233 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:49:31.314233 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:49:31.314233 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:49:31.314233 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:49:31.314233 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:49:31.314233 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:49:31.314233 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:49:31.314233 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:49:31.314233 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:49:31.314233 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:49:31.314233 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 15:49:31.314233 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 15:49:31.314233 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 15:49:31.314233 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 15:49:31.315239 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:49:31.315239 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:49:31.315239 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:49:31.315239 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:49:31.315239 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:49:31.315239 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:49:31.315239 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:49:31.315239 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application with Lock( Aug 30 15:49:31.315239 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:49:31.315239 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:49:31.315239 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:49:31.315239 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:49:31.315239 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:49:31.315239 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:49:31.315239 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:49:31.315239 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:49:31.315239 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 15:49:31.316762 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 15:49:31.316762 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 15:49:31.316762 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Aug 30 15:49:31.316762 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 608/1215] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 30 15:49:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 15:49:31.320933 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5104efd0-7aa8-4430-9ca2-ad12644d9d56 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:49:31.389742 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:49:31.431009 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5104efd0-7aa8-4430-9ca2-ad12644d9d56 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:49:31.469733 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0518692-301c-4b47-a297-3d5ce2f251f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:49:31.470361 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0518692-301c-4b47-a297-3d5ce2f251f1 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:49:31.470638 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0518692-301c-4b47-a297-3d5ce2f251f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:49:31.470906 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0518692-301c-4b47-a297-3d5ce2f251f1 None admin] PATH_INFO: `/v3/domains` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:49:31.471584 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0518692-301c-4b47-a297-3d5ce2f251f1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:49:31.471879 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0518692-301c-4b47-a297-3d5ce2f251f1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:49:31.473312 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0518692-301c-4b47-a297-3d5ce2f251f1 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:49:31.485181 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 608/1216] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Aug 30 15:49:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:49:31.491355 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4458542c-f644-4a15-9981-c5311387fbc4 None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:49:31.566911 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:49:31.599728 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4458542c-f644-4a15-9981-c5311387fbc4 None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:49:31.638449 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b8890a-b87a-4715-bd75-1b7e96df20f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:49:31.642123 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b8890a-b87a-4715-bd75-1b7e96df20f4 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:49:31.642123 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b8890a-b87a-4715-bd75-1b7e96df20f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:49:31.642123 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b8890a-b87a-4715-bd75-1b7e96df20f4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:49:31.647846 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b8890a-b87a-4715-bd75-1b7e96df20f4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:49:31.649499 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b8890a-b87a-4715-bd75-1b7e96df20f4 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:49:31.662461 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application [None req-22b8890a-b87a-4715-bd75-1b7e96df20f4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 15:49:31.662461 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:49:31.662461 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:49:31.662461 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:49:31.662461 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:49:31.662461 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:49:31.662461 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:49:31.662461 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:49:31.662461 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:49:31.662461 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:49:31.662461 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:49:31.662461 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:49:31.662461 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 15:49:31.662461 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 15:49:31.662461 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 15:49:31.663445 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 15:49:31.663445 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:49:31.663445 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:49:31.663445 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:49:31.663445 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:49:31.663445 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:49:31.663445 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:49:31.663445 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:49:31.663445 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application with Lock( Aug 30 15:49:31.663445 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:49:31.663445 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:49:31.663445 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:49:31.663445 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:49:31.663445 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:49:31.663445 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:49:31.663445 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:49:31.663445 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:49:31.669862 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 15:49:31.669862 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 15:49:31.669862 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 15:49:31.669862 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 15:49:31.669862 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 15:49:31.669862 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 15:49:31.669862 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 15:49:31.669862 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Aug 30 15:49:31.669862 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 609/1217] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Aug 30 15:49:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 15:49:31.670392 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0518692-301c-4b47-a297-3d5ce2f251f1 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:49:31.796947 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0518692-301c-4b47-a297-3d5ce2f251f1 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:49:31.823501 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:49:31.836397 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d86e95e-1acf-4727-86e1-0fb60d63d55b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:49:31.837188 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d86e95e-1acf-4727-86e1-0fb60d63d55b None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:49:31.837523 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d86e95e-1acf-4727-86e1-0fb60d63d55b None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:49:31.837863 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d86e95e-1acf-4727-86e1-0fb60d63d55b None admin] PATH_INFO: `/v3/projects` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:49:31.838691 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d86e95e-1acf-4727-86e1-0fb60d63d55b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:49:31.839069 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d86e95e-1acf-4727-86e1-0fb60d63d55b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:49:31.840639 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d86e95e-1acf-4727-86e1-0fb60d63d55b None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:49:31.854026 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 609/1218] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Aug 30 15:49:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:49:31.860360 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b8890a-b87a-4715-bd75-1b7e96df20f4 None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:49:31.980464 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b8890a-b87a-4715-bd75-1b7e96df20f4 None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:49:32.012595 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:49:32.019879 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ababe458-19bf-4d38-9875-8b281a830d89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:49:32.020795 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ababe458-19bf-4d38-9875-8b281a830d89 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:49:32.021425 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ababe458-19bf-4d38-9875-8b281a830d89 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:49:32.021979 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ababe458-19bf-4d38-9875-8b281a830d89 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:49:32.023004 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ababe458-19bf-4d38-9875-8b281a830d89 None admin] RBAC: Adding query filter params (role.id=57eb8466096346b39319536c6feeb6f7, scope.project.id=854fe71b49224dff9a4b1766a20ab266, user.id=bc12c6d31eab4a17a160250adc2380c8) {{(pid=81512) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:49:32.024106 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ababe458-19bf-4d38-9875-8b281a830d89 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:49:32.026057 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ababe458-19bf-4d38-9875-8b281a830d89 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:49:32.041473 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 610/1219] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Aug 30 15:49:31 2022] GET /identity/v3/role_assignments?user.id=bc12c6d31eab4a17a160250adc2380c8&scope.project.id=854fe71b49224dff9a4b1766a20ab266&role.id=57eb8466096346b39319536c6feeb6f7 => generated 259 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 15:49:34.000871 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f56b759-9530-40a1-b53e-85f614830d77 None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:49:34.001439 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f56b759-9530-40a1-b53e-85f614830d77 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:49:34.001439 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f56b759-9530-40a1-b53e-85f614830d77 None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:49:34.002710 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 610/1220] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 30 15:49:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 15:49:34.009301 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e56ba20-0f89-4fa3-a4a4-cad3006640cf None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:49:34.009729 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e56ba20-0f89-4fa3-a4a4-cad3006640cf None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:49:34.009974 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e56ba20-0f89-4fa3-a4a4-cad3006640cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:49:34.030140 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-1e56ba20-0f89-4fa3-a4a4-cad3006640cf None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:49:34.031321 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-1e56ba20-0f89-4fa3-a4a4-cad3006640cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:49:34.105679 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 611/1221] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 30 15:49:34 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 15:49:34.400699 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f56b759-9530-40a1-b53e-85f614830d77 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:49:34.521849 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f56b759-9530-40a1-b53e-85f614830d77 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:49:34.561137 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32fe160-0ec5-431d-ac42-cd0fa7ade975 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:49:34.562043 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32fe160-0ec5-431d-ac42-cd0fa7ade975 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:49:34.562427 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32fe160-0ec5-431d-ac42-cd0fa7ade975 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:49:34.562727 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32fe160-0ec5-431d-ac42-cd0fa7ade975 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:49:34.575791 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32fe160-0ec5-431d-ac42-cd0fa7ade975 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 15:49:34.581719 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-a32fe160-0ec5-431d-ac42-cd0fa7ade975 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 15:49:34.581719 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:49:34.581719 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:49:34.581719 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:49:34.581719 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:49:34.581719 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:49:34.581719 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:49:34.581719 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:49:34.581719 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:49:34.581719 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:49:34.581719 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:49:34.581719 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:49:34.581719 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 15:49:34.581719 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 15:49:34.581719 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 15:49:34.581719 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 15:49:34.582857 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 15:49:34.582857 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 15:49:34.582857 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 15:49:34.582857 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 15:49:34.582857 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:49:34.582857 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:49:34.582857 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:49:34.582857 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:49:34.582857 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:49:34.582857 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:49:34.582857 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:49:34.582857 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Aug 30 15:49:34.582857 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:49:34.582857 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:49:34.582857 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:49:34.582857 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:49:34.583741 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:49:34.583741 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:49:34.583741 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:49:34.583741 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:49:34.583741 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 15:49:34.583741 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 15:49:34.583741 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 15:49:34.583741 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 15:49:34.583741 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 15:49:34.583741 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 15:49:34.583741 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 15:49:34.583741 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Aug 30 15:49:34.583741 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 611/1222] 10.222.0.20 () {56 vars in 1102 bytes} [Tue Aug 30 15:49:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 15:49:34.589527 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e56ba20-0f89-4fa3-a4a4-cad3006640cf None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:49:34.704048 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e56ba20-0f89-4fa3-a4a4-cad3006640cf None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:49:34.731721 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:49:34.744142 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82dd4db0-1e13-4bb6-b35b-c2cd3270c30d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:49:34.744754 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dd4db0-1e13-4bb6-b35b-c2cd3270c30d None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:49:34.745026 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dd4db0-1e13-4bb6-b35b-c2cd3270c30d None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:49:34.745326 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dd4db0-1e13-4bb6-b35b-c2cd3270c30d None admin] PATH_INFO: `/v3/roles` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:49:34.745991 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82dd4db0-1e13-4bb6-b35b-c2cd3270c30d None admin] RBAC: Adding query filter params (name=service) {{(pid=81512) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:49:34.746302 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82dd4db0-1e13-4bb6-b35b-c2cd3270c30d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:49:34.747714 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82dd4db0-1e13-4bb6-b35b-c2cd3270c30d None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:49:34.755912 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 612/1223] 10.222.0.20 () {56 vars in 1103 bytes} [Tue Aug 30 15:49:34 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:49:34.762663 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32fe160-0ec5-431d-ac42-cd0fa7ade975 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:49:34.808724 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:49:34.867190 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32fe160-0ec5-431d-ac42-cd0fa7ade975 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:49:34.900018 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed12806-d5f6-49e5-acc6-46a9616252b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:49:34.900646 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed12806-d5f6-49e5-acc6-46a9616252b3 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:49:34.900918 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed12806-d5f6-49e5-acc6-46a9616252b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:49:34.901182 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed12806-d5f6-49e5-acc6-46a9616252b3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:49:34.912387 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed12806-d5f6-49e5-acc6-46a9616252b3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:49:34.913901 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed12806-d5f6-49e5-acc6-46a9616252b3 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:49:34.931400 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-4ed12806-d5f6-49e5-acc6-46a9616252b3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 15:49:34.931400 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:49:34.931400 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:49:34.931400 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:49:34.931400 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:49:34.931400 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:49:34.931400 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:49:34.931400 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:49:34.931400 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:49:34.931400 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:49:34.931400 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:49:34.931400 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:49:34.931400 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 15:49:34.931400 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 15:49:34.931400 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 15:49:34.931400 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 15:49:34.932435 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:49:34.932435 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:49:34.932435 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:49:34.932435 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:49:34.932435 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:49:34.932435 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:49:34.932435 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:49:34.932435 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Aug 30 15:49:34.932435 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:49:34.932435 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:49:34.932435 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:49:34.932435 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:49:34.932435 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:49:34.932435 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:49:34.932435 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:49:34.932435 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:49:34.932435 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 15:49:34.933404 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 15:49:34.933404 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 15:49:34.933404 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Aug 30 15:49:34.933404 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 612/1224] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 30 15:49:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 15:49:34.939170 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82dd4db0-1e13-4bb6-b35b-c2cd3270c30d None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:49:35.055948 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82dd4db0-1e13-4bb6-b35b-c2cd3270c30d None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:49:35.090398 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8b852b-8fce-4655-85e4-d9da0cd70505 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:49:35.091047 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8b852b-8fce-4655-85e4-d9da0cd70505 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:49:35.091323 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8b852b-8fce-4655-85e4-d9da0cd70505 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:49:35.091590 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8b852b-8fce-4655-85e4-d9da0cd70505 None admin] PATH_INFO: `/v3/domains` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:49:35.092229 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8b852b-8fce-4655-85e4-d9da0cd70505 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81512) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:49:35.092528 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8b852b-8fce-4655-85e4-d9da0cd70505 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:49:35.093930 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8b852b-8fce-4655-85e4-d9da0cd70505 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:49:35.106069 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 613/1225] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Aug 30 15:49:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:49:35.111985 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed12806-d5f6-49e5-acc6-46a9616252b3 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:49:35.209675 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:49:35.217824 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed12806-d5f6-49e5-acc6-46a9616252b3 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:49:35.252711 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe803f1-7e3d-49e4-a225-c232f7b71b67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:49:35.253370 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe803f1-7e3d-49e4-a225-c232f7b71b67 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:49:35.253645 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe803f1-7e3d-49e4-a225-c232f7b71b67 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:49:35.253916 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe803f1-7e3d-49e4-a225-c232f7b71b67 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:49:35.261542 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe803f1-7e3d-49e4-a225-c232f7b71b67 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:49:35.263213 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe803f1-7e3d-49e4-a225-c232f7b71b67 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:49:35.274986 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-efe803f1-7e3d-49e4-a225-c232f7b71b67 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 30 15:49:35.274986 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:49:35.274986 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:49:35.274986 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:49:35.274986 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:49:35.274986 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:49:35.274986 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:49:35.274986 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:49:35.274986 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:49:35.274986 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:49:35.274986 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:49:35.274986 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:49:35.274986 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 15:49:35.274986 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 15:49:35.274986 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 15:49:35.274986 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 15:49:35.276079 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:49:35.276079 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:49:35.276079 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 15:49:35.276079 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 15:49:35.276079 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 15:49:35.276079 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 15:49:35.276079 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:49:35.276079 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:49:35.276079 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:49:35.276079 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:49:35.276079 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:49:35.276079 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Aug 30 15:49:35.276079 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:49:35.276079 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:49:35.276079 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:49:35.276079 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:49:35.276079 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:49:35.276079 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:49:35.277562 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:49:35.277562 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:49:35.277562 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 15:49:35.277562 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 15:49:35.277562 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 15:49:35.277562 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 15:49:35.277562 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 15:49:35.277562 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 15:49:35.277562 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 30 15:49:35.277562 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Aug 30 15:49:35.277562 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 613/1226] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 30 15:49:35 2022] GET /identity/v3/users/placement => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 15:49:35.282251 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8b852b-8fce-4655-85e4-d9da0cd70505 None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:49:35.329978 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:49:35.382310 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8b852b-8fce-4655-85e4-d9da0cd70505 None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:49:35.428663 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4016318b-c3a6-4041-b994-49512a5f78e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:49:35.429308 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4016318b-c3a6-4041-b994-49512a5f78e8 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:49:35.429603 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4016318b-c3a6-4041-b994-49512a5f78e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:49:35.429858 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4016318b-c3a6-4041-b994-49512a5f78e8 None admin] PATH_INFO: `/v3/users` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:49:35.430640 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4016318b-c3a6-4041-b994-49512a5f78e8 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81512) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:49:35.431007 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4016318b-c3a6-4041-b994-49512a5f78e8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:49:35.433127 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4016318b-c3a6-4041-b994-49512a5f78e8 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:49:35.452276 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 614/1227] 10.222.0.20 () {56 vars in 1143 bytes} [Tue Aug 30 15:49:35 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:49:35.458285 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe803f1-7e3d-49e4-a225-c232f7b71b67 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:49:35.566797 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe803f1-7e3d-49e4-a225-c232f7b71b67 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:49:35.604544 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3010b525-815d-475d-8ca0-0e73a6908866 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:49:35.605209 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3010b525-815d-475d-8ca0-0e73a6908866 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:49:35.605532 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3010b525-815d-475d-8ca0-0e73a6908866 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:49:35.605899 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3010b525-815d-475d-8ca0-0e73a6908866 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:49:35.617031 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3010b525-815d-475d-8ca0-0e73a6908866 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:49:35.618584 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3010b525-815d-475d-8ca0-0e73a6908866 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:49:35.634080 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-3010b525-815d-475d-8ca0-0e73a6908866 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 15:49:35.634080 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:49:35.634080 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:49:35.634080 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:49:35.634080 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:49:35.634080 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:49:35.634080 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:49:35.634080 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:49:35.634080 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:49:35.634080 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:49:35.634080 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:49:35.634080 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:49:35.634080 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 15:49:35.634080 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 15:49:35.634080 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 15:49:35.634080 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 15:49:35.635285 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:49:35.635285 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:49:35.635285 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:49:35.635285 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:49:35.635285 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:49:35.635285 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:49:35.635285 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:49:35.635285 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Aug 30 15:49:35.635285 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:49:35.635285 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:49:35.635285 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:49:35.635285 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:49:35.635285 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:49:35.635285 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:49:35.635285 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:49:35.635285 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:49:35.635285 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 15:49:35.636328 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 15:49:35.636328 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 15:49:35.636328 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Aug 30 15:49:35.636328 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 614/1228] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 30 15:49:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 15:49:35.642211 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4016318b-c3a6-4041-b994-49512a5f78e8 None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:49:35.755092 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4016318b-c3a6-4041-b994-49512a5f78e8 None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:49:35.757475 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:49:35.790647 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a419927d-8b58-4692-a2c3-29c8fa05a8c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:49:35.791294 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a419927d-8b58-4692-a2c3-29c8fa05a8c7 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:49:35.791580 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a419927d-8b58-4692-a2c3-29c8fa05a8c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:49:35.791883 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a419927d-8b58-4692-a2c3-29c8fa05a8c7 None admin] PATH_INFO: `/v3/domains` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:49:35.792622 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a419927d-8b58-4692-a2c3-29c8fa05a8c7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81512) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:49:35.792911 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a419927d-8b58-4692-a2c3-29c8fa05a8c7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:49:35.794564 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a419927d-8b58-4692-a2c3-29c8fa05a8c7 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:49:35.807190 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 615/1229] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Aug 30 15:49:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:49:35.813275 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3010b525-815d-475d-8ca0-0e73a6908866 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:49:35.828727 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:49:35.926180 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3010b525-815d-475d-8ca0-0e73a6908866 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:49:35.961927 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0aeb7b-29bc-457e-b5b0-02061a0d8fec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:49:35.962548 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0aeb7b-29bc-457e-b5b0-02061a0d8fec None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:49:35.962824 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0aeb7b-29bc-457e-b5b0-02061a0d8fec None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:49:35.963092 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0aeb7b-29bc-457e-b5b0-02061a0d8fec None admin] PATH_INFO: `/v3/projects/service` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:49:35.969852 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0aeb7b-29bc-457e-b5b0-02061a0d8fec None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:49:35.971305 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0aeb7b-29bc-457e-b5b0-02061a0d8fec None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:49:35.984375 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-2a0aeb7b-29bc-457e-b5b0-02061a0d8fec None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 15:49:35.984375 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:49:35.984375 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:49:35.984375 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:49:35.984375 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:49:35.984375 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:49:35.984375 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:49:35.984375 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:49:35.984375 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:49:35.984375 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:49:35.984375 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:49:35.984375 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:49:35.984375 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 15:49:35.984375 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 15:49:35.984375 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 15:49:35.985603 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 15:49:35.985603 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:49:35.985603 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:49:35.985603 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:49:35.985603 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:49:35.985603 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:49:35.985603 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:49:35.985603 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:49:35.985603 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Aug 30 15:49:35.985603 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:49:35.985603 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:49:35.985603 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:49:35.985603 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:49:35.985603 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:49:35.985603 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:49:35.985603 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:49:35.985603 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:49:35.986718 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 15:49:35.986718 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 15:49:35.986718 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 15:49:35.986718 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 15:49:35.986718 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 15:49:35.986718 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 15:49:35.986718 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 15:49:35.986718 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Aug 30 15:49:35.986718 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 615/1230] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Aug 30 15:49:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 15:49:35.992605 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a419927d-8b58-4692-a2c3-29c8fa05a8c7 None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:49:36.102451 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a419927d-8b58-4692-a2c3-29c8fa05a8c7 None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:49:36.138122 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f12d70d-3c59-4c3b-9c6e-e49952371ab9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:49:36.138729 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f12d70d-3c59-4c3b-9c6e-e49952371ab9 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:49:36.139006 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f12d70d-3c59-4c3b-9c6e-e49952371ab9 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:49:36.139274 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f12d70d-3c59-4c3b-9c6e-e49952371ab9 None admin] PATH_INFO: `/v3/projects` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:49:36.139983 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f12d70d-3c59-4c3b-9c6e-e49952371ab9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81512) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:49:36.140277 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f12d70d-3c59-4c3b-9c6e-e49952371ab9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:49:36.141773 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f12d70d-3c59-4c3b-9c6e-e49952371ab9 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:49:36.155099 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 616/1231] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Aug 30 15:49:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:49:36.162216 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0aeb7b-29bc-457e-b5b0-02061a0d8fec None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:49:36.238590 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:49:36.275277 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0aeb7b-29bc-457e-b5b0-02061a0d8fec None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:49:36.309115 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e6676d-1d75-473d-9224-f7ae4198bbad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:49:36.309799 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e6676d-1d75-473d-9224-f7ae4198bbad None admin] REQUEST_METHOD: `PUT` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:49:36.310072 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e6676d-1d75-473d-9224-f7ae4198bbad None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:49:36.310338 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e6676d-1d75-473d-9224-f7ae4198bbad None admin] PATH_INFO: `/v3/projects/854fe71b49224dff9a4b1766a20ab266/users/bc12c6d31eab4a17a160250adc2380c8/roles/57eb8466096346b39319536c6feeb6f7` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:49:36.326431 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e6676d-1d75-473d-9224-f7ae4198bbad None admin] RBAC: Authorizing `identity:create_grant(project_id=854fe71b49224dff9a4b1766a20ab266, user_id=bc12c6d31eab4a17a160250adc2380c8, role_id=57eb8466096346b39319536c6feeb6f7)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:49:36.327896 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e6676d-1d75-473d-9224-f7ae4198bbad None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:49:36.340828 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 616/1232] 10.222.0.20 () {58 vars in 1439 bytes} [Tue Aug 30 15:49:36 2022] PUT /identity/v3/projects/854fe71b49224dff9a4b1766a20ab266/users/bc12c6d31eab4a17a160250adc2380c8/roles/57eb8466096346b39319536c6feeb6f7 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:49:38.322808 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6de445-9041-408a-a117-e9037f40f8ab None None] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:49:38.323389 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6de445-9041-408a-a117-e9037f40f8ab None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:49:38.323389 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6de445-9041-408a-a117-e9037f40f8ab None None] PATH_INFO: `/` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:49:38.324869 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 617/1233] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 30 15:49:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 15:49:38.330322 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bff1027-0112-4d97-a2c0-0ae0e34c2088 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:49:38.330721 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bff1027-0112-4d97-a2c0-0ae0e34c2088 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:49:38.330850 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bff1027-0112-4d97-a2c0-0ae0e34c2088 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:49:38.349745 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-3bff1027-0112-4d97-a2c0-0ae0e34c2088 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:49:38.350924 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-3bff1027-0112-4d97-a2c0-0ae0e34c2088 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:49:38.420416 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 617/1234] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 30 15:49:38 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 15:49:38.693167 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6de445-9041-408a-a117-e9037f40f8ab None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:49:38.719809 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:49:38.799799 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6de445-9041-408a-a117-e9037f40f8ab None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:49:38.833828 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f357fe1-c1bf-4dc3-a273-56278e497b24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:49:38.834454 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f357fe1-c1bf-4dc3-a273-56278e497b24 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:49:38.834727 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f357fe1-c1bf-4dc3-a273-56278e497b24 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:49:38.834989 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f357fe1-c1bf-4dc3-a273-56278e497b24 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:49:38.844969 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f357fe1-c1bf-4dc3-a273-56278e497b24 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 15:49:38.849917 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application [None req-2f357fe1-c1bf-4dc3-a273-56278e497b24 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 15:49:38.849917 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:49:38.849917 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:49:38.849917 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:49:38.849917 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:49:38.849917 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:49:38.849917 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:49:38.849917 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:49:38.849917 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:49:38.849917 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:49:38.849917 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:49:38.849917 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:49:38.849917 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 15:49:38.849917 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 15:49:38.849917 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 15:49:38.849917 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 15:49:38.850864 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 15:49:38.850864 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 15:49:38.850864 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 15:49:38.850864 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 15:49:38.850864 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:49:38.850864 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:49:38.850864 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:49:38.850864 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:49:38.850864 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:49:38.850864 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:49:38.850864 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:49:38.850864 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application with Lock( Aug 30 15:49:38.850864 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:49:38.850864 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:49:38.850864 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:49:38.850864 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:49:38.851724 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:49:38.851724 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:49:38.851724 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:49:38.851724 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:49:38.851724 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 15:49:38.851724 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 15:49:38.851724 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 15:49:38.851724 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 15:49:38.851724 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 15:49:38.851724 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 15:49:38.851724 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 15:49:38.851724 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Aug 30 15:49:38.851724 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 618/1235] 10.222.0.20 () {56 vars in 1102 bytes} [Tue Aug 30 15:49:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 15:49:38.859435 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bff1027-0112-4d97-a2c0-0ae0e34c2088 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:49:38.908485 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:49:38.966709 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bff1027-0112-4d97-a2c0-0ae0e34c2088 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:49:39.008095 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f3e167-5bb7-4bb6-964b-f1d4675b3270 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:49:39.008869 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f3e167-5bb7-4bb6-964b-f1d4675b3270 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:49:39.009186 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f3e167-5bb7-4bb6-964b-f1d4675b3270 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:49:39.009551 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f3e167-5bb7-4bb6-964b-f1d4675b3270 None admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:49:39.010475 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f3e167-5bb7-4bb6-964b-f1d4675b3270 None admin] RBAC: Adding query filter params (name=service) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:49:39.010851 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f3e167-5bb7-4bb6-964b-f1d4675b3270 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:49:39.012501 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f3e167-5bb7-4bb6-964b-f1d4675b3270 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:49:39.022656 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 618/1236] 10.222.0.20 () {56 vars in 1103 bytes} [Tue Aug 30 15:49:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:49:39.028935 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f357fe1-c1bf-4dc3-a273-56278e497b24 None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:49:39.120214 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:49:39.139444 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f357fe1-c1bf-4dc3-a273-56278e497b24 None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:49:39.174934 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f254b3b-516a-4ce2-a300-0874daac8ce7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:49:39.175438 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f254b3b-516a-4ce2-a300-0874daac8ce7 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:49:39.175726 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f254b3b-516a-4ce2-a300-0874daac8ce7 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:49:39.175984 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f254b3b-516a-4ce2-a300-0874daac8ce7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:49:39.186995 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f254b3b-516a-4ce2-a300-0874daac8ce7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:49:39.188494 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f254b3b-516a-4ce2-a300-0874daac8ce7 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:49:39.204074 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application [None req-5f254b3b-516a-4ce2-a300-0874daac8ce7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 15:49:39.204074 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:49:39.204074 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:49:39.204074 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:49:39.204074 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:49:39.204074 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:49:39.204074 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:49:39.204074 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:49:39.204074 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:49:39.204074 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:49:39.204074 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:49:39.204074 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:49:39.204074 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 15:49:39.204074 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 15:49:39.204074 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 15:49:39.204074 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 15:49:39.205107 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:49:39.205107 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:49:39.205107 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:49:39.205107 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:49:39.205107 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:49:39.205107 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:49:39.205107 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:49:39.205107 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application with Lock( Aug 30 15:49:39.205107 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:49:39.205107 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:49:39.205107 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:49:39.205107 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:49:39.205107 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:49:39.205107 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:49:39.205107 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:49:39.205107 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:49:39.205107 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 15:49:39.206312 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 15:49:39.206312 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 15:49:39.206312 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Aug 30 15:49:39.206312 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 619/1237] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 30 15:49:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 15:49:39.211819 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f3e167-5bb7-4bb6-964b-f1d4675b3270 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:49:39.318251 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f3e167-5bb7-4bb6-964b-f1d4675b3270 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:49:39.327720 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:49:39.354023 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec044e9-3e79-4d1c-ace9-cdb2135e944f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:49:39.354653 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec044e9-3e79-4d1c-ace9-cdb2135e944f None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:49:39.354919 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec044e9-3e79-4d1c-ace9-cdb2135e944f None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:49:39.355215 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec044e9-3e79-4d1c-ace9-cdb2135e944f None admin] PATH_INFO: `/v3/domains` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:49:39.355960 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec044e9-3e79-4d1c-ace9-cdb2135e944f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:49:39.356384 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec044e9-3e79-4d1c-ace9-cdb2135e944f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:49:39.357857 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec044e9-3e79-4d1c-ace9-cdb2135e944f None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:49:39.370408 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 619/1238] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Aug 30 15:49:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:49:39.376405 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f254b3b-516a-4ce2-a300-0874daac8ce7 None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:49:39.490132 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f254b3b-516a-4ce2-a300-0874daac8ce7 None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:49:39.529872 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8ab3d1-9474-481b-ad6b-59057d31dbeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:49:39.530523 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8ab3d1-9474-481b-ad6b-59057d31dbeb None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:49:39.530790 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8ab3d1-9474-481b-ad6b-59057d31dbeb None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:49:39.531063 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8ab3d1-9474-481b-ad6b-59057d31dbeb None admin] PATH_INFO: `/v3/users/placement` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:49:39.533920 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:49:39.539742 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8ab3d1-9474-481b-ad6b-59057d31dbeb None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:49:39.541799 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8ab3d1-9474-481b-ad6b-59057d31dbeb None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:49:39.554729 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application [None req-2b8ab3d1-9474-481b-ad6b-59057d31dbeb None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 30 15:49:39.554729 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:49:39.554729 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:49:39.554729 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:49:39.554729 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:49:39.554729 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:49:39.554729 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:49:39.554729 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:49:39.554729 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:49:39.554729 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:49:39.554729 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:49:39.554729 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:49:39.554729 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 15:49:39.554729 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 15:49:39.554729 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 15:49:39.554729 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 15:49:39.560878 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:49:39.560878 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:49:39.560878 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 15:49:39.560878 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 15:49:39.560878 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 15:49:39.560878 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 15:49:39.560878 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:49:39.560878 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:49:39.560878 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:49:39.560878 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:49:39.560878 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:49:39.560878 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application with Lock( Aug 30 15:49:39.560878 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:49:39.560878 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:49:39.560878 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:49:39.560878 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:49:39.560878 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:49:39.560878 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:49:39.562092 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:49:39.562092 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:49:39.562092 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 15:49:39.562092 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 15:49:39.562092 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 15:49:39.562092 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 15:49:39.562092 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 15:49:39.562092 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 15:49:39.562092 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 30 15:49:39.562092 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Aug 30 15:49:39.562092 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 620/1239] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 30 15:49:39 2022] GET /identity/v3/users/placement => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 15:49:39.562821 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec044e9-3e79-4d1c-ace9-cdb2135e944f None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:49:39.670760 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec044e9-3e79-4d1c-ace9-cdb2135e944f None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:49:39.708034 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b43f41c-c823-4fe9-a7a6-a895354cd0fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:49:39.708640 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b43f41c-c823-4fe9-a7a6-a895354cd0fa None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:49:39.708913 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b43f41c-c823-4fe9-a7a6-a895354cd0fa None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:49:39.709178 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b43f41c-c823-4fe9-a7a6-a895354cd0fa None admin] PATH_INFO: `/v3/users` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:49:39.709936 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b43f41c-c823-4fe9-a7a6-a895354cd0fa None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:49:39.710290 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b43f41c-c823-4fe9-a7a6-a895354cd0fa None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:49:39.712004 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b43f41c-c823-4fe9-a7a6-a895354cd0fa None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:49:39.731305 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 620/1240] 10.222.0.20 () {56 vars in 1143 bytes} [Tue Aug 30 15:49:39 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:49:39.737661 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8ab3d1-9474-481b-ad6b-59057d31dbeb None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:49:39.847050 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8ab3d1-9474-481b-ad6b-59057d31dbeb None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:49:39.881961 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbeafcd7-7149-470d-b5de-b72216a6f9e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:49:39.882554 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbeafcd7-7149-470d-b5de-b72216a6f9e0 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:49:39.882859 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbeafcd7-7149-470d-b5de-b72216a6f9e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:49:39.883130 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbeafcd7-7149-470d-b5de-b72216a6f9e0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:49:39.950614 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:49:39.957324 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbeafcd7-7149-470d-b5de-b72216a6f9e0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:49:39.958867 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbeafcd7-7149-470d-b5de-b72216a6f9e0 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:49:40.068848 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 621/1241] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Aug 30 15:49:39 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 30 15:49:40.075314 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b43f41c-c823-4fe9-a7a6-a895354cd0fa None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:49:40.104337 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:49:40.185736 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b43f41c-c823-4fe9-a7a6-a895354cd0fa None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:49:40.226310 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbbbe7b-9a73-4964-b770-cb817fd91957 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:49:40.226941 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbbbe7b-9a73-4964-b770-cb817fd91957 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:49:40.227257 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbbbe7b-9a73-4964-b770-cb817fd91957 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:49:40.227521 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbbbe7b-9a73-4964-b770-cb817fd91957 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:49:40.238849 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbbbe7b-9a73-4964-b770-cb817fd91957 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:49:40.240423 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbbbe7b-9a73-4964-b770-cb817fd91957 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:49:40.255303 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-4bbbbe7b-9a73-4964-b770-cb817fd91957 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 15:49:40.255303 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:49:40.255303 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:49:40.255303 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:49:40.255303 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:49:40.255303 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:49:40.255303 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:49:40.255303 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:49:40.255303 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:49:40.255303 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:49:40.255303 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:49:40.255303 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:49:40.255303 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 15:49:40.255303 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 15:49:40.255303 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 15:49:40.255303 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 15:49:40.256484 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:49:40.256484 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:49:40.256484 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:49:40.256484 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:49:40.256484 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:49:40.256484 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:49:40.256484 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:49:40.256484 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Aug 30 15:49:40.256484 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:49:40.256484 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:49:40.256484 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:49:40.256484 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:49:40.256484 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:49:40.256484 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:49:40.256484 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:49:40.256484 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:49:40.256484 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 15:49:40.257607 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 15:49:40.257607 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 15:49:40.257607 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Aug 30 15:49:40.257607 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 621/1242] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 30 15:49:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 15:49:40.262632 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbeafcd7-7149-470d-b5de-b72216a6f9e0 None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:49:40.362017 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:49:40.369585 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbeafcd7-7149-470d-b5de-b72216a6f9e0 None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:49:40.404483 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbf9dc1-3e98-415f-ab1b-95e5cf1260e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:49:40.405192 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbf9dc1-3e98-415f-ab1b-95e5cf1260e9 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:49:40.405432 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbf9dc1-3e98-415f-ab1b-95e5cf1260e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:49:40.406404 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbf9dc1-3e98-415f-ab1b-95e5cf1260e9 None admin] PATH_INFO: `/v3/domains` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:49:40.407167 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbf9dc1-3e98-415f-ab1b-95e5cf1260e9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81512) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:49:40.407471 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbf9dc1-3e98-415f-ab1b-95e5cf1260e9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:49:40.408858 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbf9dc1-3e98-415f-ab1b-95e5cf1260e9 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:49:40.420403 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 622/1243] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Aug 30 15:49:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:49:40.427121 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbbbe7b-9a73-4964-b770-cb817fd91957 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:49:40.511341 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:49:40.535129 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbbbe7b-9a73-4964-b770-cb817fd91957 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:49:40.572965 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b4c0c7-889a-477d-a0bd-fb7e099de4b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:49:40.574172 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b4c0c7-889a-477d-a0bd-fb7e099de4b2 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:49:40.574767 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b4c0c7-889a-477d-a0bd-fb7e099de4b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:49:40.575227 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b4c0c7-889a-477d-a0bd-fb7e099de4b2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:49:40.585561 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b4c0c7-889a-477d-a0bd-fb7e099de4b2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:49:40.587452 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b4c0c7-889a-477d-a0bd-fb7e099de4b2 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:49:40.603518 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-e3b4c0c7-889a-477d-a0bd-fb7e099de4b2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 15:49:40.603518 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:49:40.603518 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:49:40.603518 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:49:40.603518 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:49:40.603518 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:49:40.603518 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:49:40.603518 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:49:40.603518 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:49:40.603518 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:49:40.603518 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:49:40.603518 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:49:40.603518 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 15:49:40.603518 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 15:49:40.603518 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 15:49:40.604601 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 15:49:40.604601 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:49:40.604601 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:49:40.604601 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:49:40.604601 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:49:40.604601 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:49:40.604601 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:49:40.604601 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:49:40.604601 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Aug 30 15:49:40.604601 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:49:40.604601 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:49:40.604601 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:49:40.604601 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:49:40.604601 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:49:40.604601 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:49:40.604601 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:49:40.604601 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:49:40.605667 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 15:49:40.605667 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 15:49:40.605667 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 15:49:40.605667 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 15:49:40.605667 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 15:49:40.605667 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 15:49:40.605667 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 15:49:40.605667 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Aug 30 15:49:40.605667 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 622/1244] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Aug 30 15:49:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 15:49:40.612469 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbf9dc1-3e98-415f-ab1b-95e5cf1260e9 None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:49:40.728520 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbf9dc1-3e98-415f-ab1b-95e5cf1260e9 None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:49:40.764335 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd14ce8c-6442-4adb-a1d3-ce4f21329922 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:49:40.764938 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd14ce8c-6442-4adb-a1d3-ce4f21329922 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:49:40.765294 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd14ce8c-6442-4adb-a1d3-ce4f21329922 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:49:40.765508 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd14ce8c-6442-4adb-a1d3-ce4f21329922 None admin] PATH_INFO: `/v3/projects` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:49:40.766463 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd14ce8c-6442-4adb-a1d3-ce4f21329922 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81512) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:49:40.766846 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd14ce8c-6442-4adb-a1d3-ce4f21329922 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:49:40.768416 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd14ce8c-6442-4adb-a1d3-ce4f21329922 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:49:40.783128 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 623/1245] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Aug 30 15:49:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:49:40.789715 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b4c0c7-889a-477d-a0bd-fb7e099de4b2 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:49:40.896708 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b4c0c7-889a-477d-a0bd-fb7e099de4b2 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:49:40.933022 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77360457-ec52-4b89-af53-41a2d11575c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:49:40.933756 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77360457-ec52-4b89-af53-41a2d11575c2 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:49:40.934064 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77360457-ec52-4b89-af53-41a2d11575c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:49:40.934365 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77360457-ec52-4b89-af53-41a2d11575c2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:49:40.935187 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77360457-ec52-4b89-af53-41a2d11575c2 None admin] RBAC: Adding query filter params (role.id=57eb8466096346b39319536c6feeb6f7, scope.project.id=854fe71b49224dff9a4b1766a20ab266, user.id=bc12c6d31eab4a17a160250adc2380c8) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:49:40.935556 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77360457-ec52-4b89-af53-41a2d11575c2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:49:40.937131 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77360457-ec52-4b89-af53-41a2d11575c2 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:49:40.943405 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 623/1246] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Aug 30 15:49:40 2022] GET /identity/v3/role_assignments?user.id=bc12c6d31eab4a17a160250adc2380c8&scope.project.id=854fe71b49224dff9a4b1766a20ab266&role.id=57eb8466096346b39319536c6feeb6f7 => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:49:42.906395 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805d333f-c390-42fb-a861-e37001b43e01 None None] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:49:42.907078 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805d333f-c390-42fb-a861-e37001b43e01 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:49:42.907078 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805d333f-c390-42fb-a861-e37001b43e01 None None] PATH_INFO: `/` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:49:42.908208 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 624/1247] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 30 15:49:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 15:49:42.913588 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272df139-5380-4aae-8063-e415bf4ea644 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:49:42.914119 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272df139-5380-4aae-8063-e415bf4ea644 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:49:42.914119 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272df139-5380-4aae-8063-e415bf4ea644 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:49:42.920892 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:49:42.935791 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-272df139-5380-4aae-8063-e415bf4ea644 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:49:42.937175 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-272df139-5380-4aae-8063-e415bf4ea644 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:49:43.014020 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 624/1248] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 30 15:49:42 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 15:49:43.275490 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805d333f-c390-42fb-a861-e37001b43e01 None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:49:43.294399 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:49:43.393391 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805d333f-c390-42fb-a861-e37001b43e01 None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:49:43.428443 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9158a842-860a-4f95-adc0-9bd6c6a6fb73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:49:43.429101 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9158a842-860a-4f95-adc0-9bd6c6a6fb73 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:49:43.429579 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9158a842-860a-4f95-adc0-9bd6c6a6fb73 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:49:43.429897 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9158a842-860a-4f95-adc0-9bd6c6a6fb73 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:49:43.442210 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9158a842-860a-4f95-adc0-9bd6c6a6fb73 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 15:49:43.449032 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application [None req-9158a842-860a-4f95-adc0-9bd6c6a6fb73 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 15:49:43.449032 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:49:43.449032 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:49:43.449032 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:49:43.449032 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:49:43.449032 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:49:43.449032 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:49:43.449032 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:49:43.449032 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:49:43.449032 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:49:43.449032 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:49:43.449032 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:49:43.449032 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 15:49:43.449032 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 15:49:43.449032 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 15:49:43.449032 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 15:49:43.450610 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 15:49:43.450610 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 15:49:43.450610 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 15:49:43.450610 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 15:49:43.450610 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:49:43.450610 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:49:43.450610 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:49:43.450610 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:49:43.450610 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:49:43.450610 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:49:43.450610 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:49:43.450610 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application with Lock( Aug 30 15:49:43.450610 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:49:43.450610 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:49:43.450610 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:49:43.450610 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:49:43.451500 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:49:43.451500 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:49:43.451500 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:49:43.451500 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:49:43.451500 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 15:49:43.451500 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 15:49:43.451500 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 15:49:43.451500 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 15:49:43.451500 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 15:49:43.451500 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 15:49:43.451500 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 15:49:43.451500 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Aug 30 15:49:43.451500 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 625/1249] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Aug 30 15:49:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 15:49:43.457608 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272df139-5380-4aae-8063-e415bf4ea644 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:49:43.587118 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272df139-5380-4aae-8063-e415bf4ea644 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:49:43.617732 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:49:43.623318 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5ec80b-3097-4914-99ac-192f1f3b0865 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:49:43.623944 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5ec80b-3097-4914-99ac-192f1f3b0865 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:49:43.624221 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5ec80b-3097-4914-99ac-192f1f3b0865 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:49:43.624494 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5ec80b-3097-4914-99ac-192f1f3b0865 None admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:49:43.625167 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5ec80b-3097-4914-99ac-192f1f3b0865 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:49:43.625499 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5ec80b-3097-4914-99ac-192f1f3b0865 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:49:43.626895 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5ec80b-3097-4914-99ac-192f1f3b0865 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:49:43.639399 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 625/1250] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Aug 30 15:49:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:49:43.646227 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9158a842-860a-4f95-adc0-9bd6c6a6fb73 None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:49:43.731952 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:49:43.758191 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9158a842-860a-4f95-adc0-9bd6c6a6fb73 None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:49:43.800549 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5766e195-6839-4f57-b383-98ebbb052d84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:49:43.801206 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5766e195-6839-4f57-b383-98ebbb052d84 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:49:43.801831 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5766e195-6839-4f57-b383-98ebbb052d84 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:49:43.802104 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5766e195-6839-4f57-b383-98ebbb052d84 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:49:43.816127 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5766e195-6839-4f57-b383-98ebbb052d84 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:49:43.818543 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5766e195-6839-4f57-b383-98ebbb052d84 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:49:43.837310 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application [None req-5766e195-6839-4f57-b383-98ebbb052d84 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 15:49:43.837310 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:49:43.837310 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:49:43.837310 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:49:43.837310 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:49:43.837310 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:49:43.837310 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:49:43.837310 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:49:43.837310 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:49:43.837310 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:49:43.837310 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:49:43.837310 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:49:43.837310 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 15:49:43.837310 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 15:49:43.837310 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 15:49:43.837310 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 15:49:43.838503 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:49:43.838503 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:49:43.838503 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:49:43.838503 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:49:43.838503 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:49:43.838503 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:49:43.838503 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:49:43.838503 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application with Lock( Aug 30 15:49:43.838503 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:49:43.838503 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:49:43.838503 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:49:43.838503 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:49:43.838503 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:49:43.838503 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:49:43.838503 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:49:43.838503 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:49:43.838503 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 15:49:43.839610 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 15:49:43.839610 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 15:49:43.839610 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Aug 30 15:49:43.839610 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 626/1251] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 30 15:49:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 15:49:43.845890 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5ec80b-3097-4914-99ac-192f1f3b0865 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:49:43.963309 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5ec80b-3097-4914-99ac-192f1f3b0865 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:49:44.004040 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915f3de3-3d19-4040-9a0b-5117bed4a977 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:49:44.004953 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915f3de3-3d19-4040-9a0b-5117bed4a977 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:49:44.005360 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915f3de3-3d19-4040-9a0b-5117bed4a977 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:49:44.005640 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915f3de3-3d19-4040-9a0b-5117bed4a977 None admin] PATH_INFO: `/v3/domains` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:49:44.006483 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915f3de3-3d19-4040-9a0b-5117bed4a977 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:49:44.006876 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915f3de3-3d19-4040-9a0b-5117bed4a977 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:49:44.008591 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915f3de3-3d19-4040-9a0b-5117bed4a977 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:49:44.023692 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 626/1252] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Aug 30 15:49:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:49:44.030825 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5766e195-6839-4f57-b383-98ebbb052d84 None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:49:44.141367 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5766e195-6839-4f57-b383-98ebbb052d84 None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:49:44.169036 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:49:44.179091 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67876b4-9038-4f3c-8a87-0c35c48644b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:49:44.179738 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67876b4-9038-4f3c-8a87-0c35c48644b3 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:49:44.180016 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67876b4-9038-4f3c-8a87-0c35c48644b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:49:44.180301 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67876b4-9038-4f3c-8a87-0c35c48644b3 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:49:44.189932 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67876b4-9038-4f3c-8a87-0c35c48644b3 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:49:44.191416 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67876b4-9038-4f3c-8a87-0c35c48644b3 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:49:44.204626 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application [None req-d67876b4-9038-4f3c-8a87-0c35c48644b3 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 30 15:49:44.204626 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:49:44.204626 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:49:44.204626 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:49:44.204626 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:49:44.204626 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:49:44.204626 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:49:44.204626 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:49:44.204626 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:49:44.204626 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:49:44.204626 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:49:44.204626 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:49:44.204626 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 15:49:44.204626 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 15:49:44.204626 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 15:49:44.204626 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 15:49:44.205868 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:49:44.205868 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:49:44.205868 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 15:49:44.205868 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 15:49:44.205868 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 15:49:44.205868 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 15:49:44.205868 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:49:44.205868 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:49:44.205868 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:49:44.205868 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:49:44.205868 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:49:44.205868 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application with Lock( Aug 30 15:49:44.205868 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:49:44.205868 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:49:44.205868 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:49:44.205868 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:49:44.205868 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:49:44.205868 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:49:44.207236 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:49:44.207236 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:49:44.207236 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 15:49:44.207236 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 15:49:44.207236 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 15:49:44.207236 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 15:49:44.207236 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 15:49:44.207236 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 15:49:44.207236 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 30 15:49:44.207236 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Aug 30 15:49:44.207236 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 627/1253] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 30 15:49:44 2022] GET /identity/v3/users/placement => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 15:49:44.212232 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915f3de3-3d19-4040-9a0b-5117bed4a977 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:49:44.274120 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:49:44.323634 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915f3de3-3d19-4040-9a0b-5117bed4a977 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:49:44.361914 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65975dec-2ee2-4da5-a459-3969b0089321 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:49:44.362524 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65975dec-2ee2-4da5-a459-3969b0089321 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:49:44.362794 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65975dec-2ee2-4da5-a459-3969b0089321 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:49:44.363056 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65975dec-2ee2-4da5-a459-3969b0089321 None admin] PATH_INFO: `/v3/users` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:49:44.363808 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65975dec-2ee2-4da5-a459-3969b0089321 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:49:44.364109 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65975dec-2ee2-4da5-a459-3969b0089321 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:49:44.365950 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65975dec-2ee2-4da5-a459-3969b0089321 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:49:44.386511 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 627/1254] 10.222.0.20 () {56 vars in 1143 bytes} [Tue Aug 30 15:49:44 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:49:44.392705 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67876b4-9038-4f3c-8a87-0c35c48644b3 None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:49:44.502738 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67876b4-9038-4f3c-8a87-0c35c48644b3 None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:49:44.541537 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128c30f0-7007-44d6-9044-9877bca7d60f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:49:44.542198 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128c30f0-7007-44d6-9044-9877bca7d60f None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:49:44.542566 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128c30f0-7007-44d6-9044-9877bca7d60f None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:49:44.542855 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128c30f0-7007-44d6-9044-9877bca7d60f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:49:44.604646 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:49:44.620279 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128c30f0-7007-44d6-9044-9877bca7d60f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:49:44.621831 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128c30f0-7007-44d6-9044-9877bca7d60f None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:49:44.733384 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 628/1255] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Aug 30 15:49:44 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 30 15:49:44.739388 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65975dec-2ee2-4da5-a459-3969b0089321 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:49:44.844555 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65975dec-2ee2-4da5-a459-3969b0089321 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:49:44.858574 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:49:44.881009 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5c55fc-22c2-48fc-8711-699c22257296 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:49:44.881679 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5c55fc-22c2-48fc-8711-699c22257296 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:49:44.881942 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5c55fc-22c2-48fc-8711-699c22257296 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:49:44.882247 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5c55fc-22c2-48fc-8711-699c22257296 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:49:44.892567 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5c55fc-22c2-48fc-8711-699c22257296 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:49:44.894042 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5c55fc-22c2-48fc-8711-699c22257296 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:49:44.910173 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-da5c55fc-22c2-48fc-8711-699c22257296 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 15:49:44.910173 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:49:44.910173 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:49:44.910173 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:49:44.910173 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:49:44.910173 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:49:44.910173 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:49:44.910173 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:49:44.910173 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:49:44.910173 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:49:44.910173 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:49:44.910173 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:49:44.910173 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 15:49:44.910173 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 15:49:44.910173 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 15:49:44.910173 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 15:49:44.911298 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:49:44.911298 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:49:44.911298 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:49:44.911298 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:49:44.911298 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:49:44.911298 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:49:44.911298 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:49:44.911298 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Aug 30 15:49:44.911298 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:49:44.911298 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:49:44.911298 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:49:44.911298 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:49:44.911298 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:49:44.911298 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:49:44.911298 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:49:44.911298 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:49:44.911298 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 15:49:44.912551 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 15:49:44.912551 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 15:49:44.912551 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Aug 30 15:49:44.912551 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 628/1256] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 30 15:49:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 15:49:44.917265 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128c30f0-7007-44d6-9044-9877bca7d60f None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:49:45.014957 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:49:45.028451 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128c30f0-7007-44d6-9044-9877bca7d60f None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:49:45.066345 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7eadd26-bdda-4936-88f1-8eb5571b7ea7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:49:45.067080 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7eadd26-bdda-4936-88f1-8eb5571b7ea7 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:49:45.067186 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7eadd26-bdda-4936-88f1-8eb5571b7ea7 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:49:45.067455 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7eadd26-bdda-4936-88f1-8eb5571b7ea7 None admin] PATH_INFO: `/v3/domains` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:49:45.068089 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7eadd26-bdda-4936-88f1-8eb5571b7ea7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81512) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:49:45.068432 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7eadd26-bdda-4936-88f1-8eb5571b7ea7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:49:45.069902 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7eadd26-bdda-4936-88f1-8eb5571b7ea7 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:49:45.081112 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 629/1257] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Aug 30 15:49:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:49:45.086933 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5c55fc-22c2-48fc-8711-699c22257296 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:49:45.197110 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5c55fc-22c2-48fc-8711-699c22257296 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:49:45.233900 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a36fd7-559c-4279-8d28-c7d404db4a4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:49:45.234547 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a36fd7-559c-4279-8d28-c7d404db4a4d None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:49:45.234859 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a36fd7-559c-4279-8d28-c7d404db4a4d None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:49:45.235126 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a36fd7-559c-4279-8d28-c7d404db4a4d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:49:45.242600 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a36fd7-559c-4279-8d28-c7d404db4a4d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:49:45.244169 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a36fd7-559c-4279-8d28-c7d404db4a4d None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:49:45.256045 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-f9a36fd7-559c-4279-8d28-c7d404db4a4d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 15:49:45.256045 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:49:45.256045 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:49:45.256045 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:49:45.256045 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:49:45.256045 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:49:45.256045 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:49:45.256045 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:49:45.256045 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:49:45.256045 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:49:45.256045 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:49:45.256045 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:49:45.256045 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 15:49:45.256045 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 15:49:45.256045 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 15:49:45.257172 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 15:49:45.257172 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:49:45.257172 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:49:45.257172 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:49:45.257172 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:49:45.257172 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:49:45.257172 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:49:45.257172 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:49:45.257172 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Aug 30 15:49:45.257172 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:49:45.257172 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:49:45.257172 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:49:45.257172 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:49:45.257172 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:49:45.257172 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:49:45.257172 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:49:45.257172 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:49:45.258310 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 15:49:45.258310 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 15:49:45.258310 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 15:49:45.258310 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 15:49:45.258310 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 15:49:45.258310 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 15:49:45.258310 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 15:49:45.258310 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Aug 30 15:49:45.258310 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 629/1258] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Aug 30 15:49:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 15:49:45.263688 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7eadd26-bdda-4936-88f1-8eb5571b7ea7 None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:49:45.371541 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7eadd26-bdda-4936-88f1-8eb5571b7ea7 None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:49:45.408215 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacece58-cdbb-4b87-bee5-51e5ee81cfc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:49:45.408842 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacece58-cdbb-4b87-bee5-51e5ee81cfc7 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:49:45.409119 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacece58-cdbb-4b87-bee5-51e5ee81cfc7 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:49:45.409492 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacece58-cdbb-4b87-bee5-51e5ee81cfc7 None admin] PATH_INFO: `/v3/projects` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:49:45.410194 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aacece58-cdbb-4b87-bee5-51e5ee81cfc7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81512) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:49:45.410504 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aacece58-cdbb-4b87-bee5-51e5ee81cfc7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:49:45.412069 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aacece58-cdbb-4b87-bee5-51e5ee81cfc7 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:49:45.425847 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 630/1259] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Aug 30 15:49:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:49:45.432893 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a36fd7-559c-4279-8d28-c7d404db4a4d None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:49:45.460760 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:49:45.546464 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a36fd7-559c-4279-8d28-c7d404db4a4d None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:49:45.582242 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0dfdc7-73df-4d90-aa97-9fdcb195098e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:49:45.582842 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0dfdc7-73df-4d90-aa97-9fdcb195098e None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:49:45.583114 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0dfdc7-73df-4d90-aa97-9fdcb195098e None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:49:45.583379 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0dfdc7-73df-4d90-aa97-9fdcb195098e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:49:45.584094 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0dfdc7-73df-4d90-aa97-9fdcb195098e None admin] RBAC: Adding query filter params (role.id=f265386950a641ac8413b53eda4ae34d, scope.project.id=854fe71b49224dff9a4b1766a20ab266, user.id=bc12c6d31eab4a17a160250adc2380c8) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:49:45.584389 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0dfdc7-73df-4d90-aa97-9fdcb195098e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:49:45.585847 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0dfdc7-73df-4d90-aa97-9fdcb195098e None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:49:45.591652 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 630/1260] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Aug 30 15:49:45 2022] GET /identity/v3/role_assignments?user.id=bc12c6d31eab4a17a160250adc2380c8&scope.project.id=854fe71b49224dff9a4b1766a20ab266&role.id=f265386950a641ac8413b53eda4ae34d => generated 259 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:49:47.463252 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2387a97-88ca-4996-88e2-0314af099041 None None] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:49:47.463820 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2387a97-88ca-4996-88e2-0314af099041 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:49:47.463820 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2387a97-88ca-4996-88e2-0314af099041 None None] PATH_INFO: `/` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:49:47.465179 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 631/1261] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 30 15:49:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 15:49:47.471690 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad88ad87-1e21-4e1d-9cec-ae3c1a495148 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:49:47.472439 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad88ad87-1e21-4e1d-9cec-ae3c1a495148 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:49:47.472574 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad88ad87-1e21-4e1d-9cec-ae3c1a495148 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:49:47.494733 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-ad88ad87-1e21-4e1d-9cec-ae3c1a495148 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:49:47.495931 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-ad88ad87-1e21-4e1d-9cec-ae3c1a495148 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:49:47.572631 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 631/1262] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 30 15:49:47 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 15:49:47.844198 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2387a97-88ca-4996-88e2-0314af099041 None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:49:47.856728 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:49:47.955011 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2387a97-88ca-4996-88e2-0314af099041 None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:49:47.993878 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547ef8ec-04ca-4472-bdfc-69e6744b345a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:49:47.994610 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547ef8ec-04ca-4472-bdfc-69e6744b345a None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:49:47.994921 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547ef8ec-04ca-4472-bdfc-69e6744b345a None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:49:47.995196 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547ef8ec-04ca-4472-bdfc-69e6744b345a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:49:48.007202 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547ef8ec-04ca-4472-bdfc-69e6744b345a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 15:49:48.011963 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application [None req-547ef8ec-04ca-4472-bdfc-69e6744b345a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 15:49:48.011963 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:49:48.011963 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:49:48.011963 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:49:48.011963 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:49:48.011963 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:49:48.011963 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:49:48.011963 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:49:48.011963 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:49:48.011963 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:49:48.011963 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:49:48.011963 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:49:48.011963 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 15:49:48.011963 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 15:49:48.011963 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 15:49:48.011963 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 15:49:48.013428 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 15:49:48.013428 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 15:49:48.013428 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 15:49:48.013428 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 15:49:48.013428 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:49:48.013428 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:49:48.013428 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:49:48.013428 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:49:48.013428 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:49:48.013428 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:49:48.013428 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:49:48.013428 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application with Lock( Aug 30 15:49:48.013428 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:49:48.013428 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:49:48.013428 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:49:48.013428 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:49:48.014529 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:49:48.014529 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:49:48.014529 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:49:48.014529 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:49:48.014529 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 15:49:48.014529 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 15:49:48.014529 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 15:49:48.014529 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 15:49:48.014529 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 15:49:48.014529 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 15:49:48.014529 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 15:49:48.014529 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Aug 30 15:49:48.014529 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 632/1263] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Aug 30 15:49:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 15:49:48.020319 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad88ad87-1e21-4e1d-9cec-ae3c1a495148 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:49:48.043373 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:49:48.133320 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad88ad87-1e21-4e1d-9cec-ae3c1a495148 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:49:48.168114 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cafca7c-a3b7-4b9d-9288-57c809e56202 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:49:48.168717 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cafca7c-a3b7-4b9d-9288-57c809e56202 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:49:48.168995 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cafca7c-a3b7-4b9d-9288-57c809e56202 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:49:48.169265 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cafca7c-a3b7-4b9d-9288-57c809e56202 None admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:49:48.169951 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cafca7c-a3b7-4b9d-9288-57c809e56202 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:49:48.170253 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cafca7c-a3b7-4b9d-9288-57c809e56202 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:49:48.171631 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cafca7c-a3b7-4b9d-9288-57c809e56202 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:49:48.179596 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 632/1264] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Aug 30 15:49:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 15:49:48.185751 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547ef8ec-04ca-4472-bdfc-69e6744b345a None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:49:48.252385 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:49:48.287039 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547ef8ec-04ca-4472-bdfc-69e6744b345a None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:49:48.322418 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f550e0e-1f74-4116-a3ba-e60a48e85495 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:49:48.323033 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f550e0e-1f74-4116-a3ba-e60a48e85495 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:49:48.323306 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f550e0e-1f74-4116-a3ba-e60a48e85495 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:49:48.323551 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f550e0e-1f74-4116-a3ba-e60a48e85495 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:49:48.333976 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f550e0e-1f74-4116-a3ba-e60a48e85495 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:49:48.335408 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f550e0e-1f74-4116-a3ba-e60a48e85495 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:49:48.349874 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application [None req-0f550e0e-1f74-4116-a3ba-e60a48e85495 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 15:49:48.349874 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:49:48.349874 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:49:48.349874 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:49:48.349874 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:49:48.349874 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:49:48.349874 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:49:48.349874 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:49:48.349874 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:49:48.349874 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:49:48.349874 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:49:48.349874 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:49:48.349874 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 15:49:48.349874 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 15:49:48.349874 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 15:49:48.349874 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 15:49:48.350937 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:49:48.350937 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:49:48.350937 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:49:48.350937 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:49:48.350937 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:49:48.350937 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:49:48.350937 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:49:48.350937 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application with Lock( Aug 30 15:49:48.350937 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:49:48.350937 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:49:48.350937 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:49:48.350937 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:49:48.350937 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:49:48.350937 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:49:48.350937 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:49:48.350937 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:49:48.350937 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 15:49:48.352196 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 15:49:48.352196 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 15:49:48.352196 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Aug 30 15:49:48.352196 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 633/1265] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 30 15:49:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 15:49:48.356850 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cafca7c-a3b7-4b9d-9288-57c809e56202 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:49:48.450615 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:49:48.471380 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cafca7c-a3b7-4b9d-9288-57c809e56202 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:49:48.506321 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb166e1-2569-407b-9ef5-c33fd6127baa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:49:48.506917 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb166e1-2569-407b-9ef5-c33fd6127baa None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:49:48.507173 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb166e1-2569-407b-9ef5-c33fd6127baa None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:49:48.507447 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb166e1-2569-407b-9ef5-c33fd6127baa None admin] PATH_INFO: `/v3/domains` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:49:48.508090 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb166e1-2569-407b-9ef5-c33fd6127baa None admin] RBAC: Adding query filter params (name=Default) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:49:48.508382 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb166e1-2569-407b-9ef5-c33fd6127baa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:49:48.509790 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb166e1-2569-407b-9ef5-c33fd6127baa None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:49:48.521485 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 633/1266] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Aug 30 15:49:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:49:48.530802 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f550e0e-1f74-4116-a3ba-e60a48e85495 None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:49:48.641287 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f550e0e-1f74-4116-a3ba-e60a48e85495 None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:49:48.661150 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:49:48.675452 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54b541b-23d4-45a7-bf0b-e882c86d165d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:49:48.676073 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54b541b-23d4-45a7-bf0b-e882c86d165d None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:49:48.676346 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54b541b-23d4-45a7-bf0b-e882c86d165d None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:49:48.676609 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54b541b-23d4-45a7-bf0b-e882c86d165d None admin] PATH_INFO: `/v3/users/placement` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:49:48.684003 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54b541b-23d4-45a7-bf0b-e882c86d165d None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:49:48.685483 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54b541b-23d4-45a7-bf0b-e882c86d165d None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:49:48.696241 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application [None req-e54b541b-23d4-45a7-bf0b-e882c86d165d None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 30 15:49:48.696241 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:49:48.696241 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:49:48.696241 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:49:48.696241 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:49:48.696241 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:49:48.696241 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:49:48.696241 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:49:48.696241 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:49:48.696241 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:49:48.696241 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:49:48.696241 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:49:48.696241 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 15:49:48.696241 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 15:49:48.696241 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 15:49:48.696241 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 15:49:48.697319 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:49:48.697319 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:49:48.697319 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 15:49:48.697319 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 15:49:48.697319 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 15:49:48.697319 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 15:49:48.697319 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:49:48.697319 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:49:48.697319 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:49:48.697319 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:49:48.697319 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:49:48.697319 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application with Lock( Aug 30 15:49:48.697319 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:49:48.697319 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:49:48.697319 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:49:48.697319 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:49:48.697319 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:49:48.697319 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:49:48.698379 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:49:48.698379 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:49:48.698379 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 15:49:48.698379 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 15:49:48.698379 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 15:49:48.698379 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 15:49:48.698379 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 15:49:48.698379 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 15:49:48.698379 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 30 15:49:48.698379 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Aug 30 15:49:48.698379 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 634/1267] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 30 15:49:48 2022] GET /identity/v3/users/placement => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 15:49:48.703493 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb166e1-2569-407b-9ef5-c33fd6127baa None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:49:48.809738 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb166e1-2569-407b-9ef5-c33fd6127baa None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:49:48.845608 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cbf2ef-fce7-44e7-8aaa-06ee1950cd18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:49:48.846201 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cbf2ef-fce7-44e7-8aaa-06ee1950cd18 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:49:48.846470 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cbf2ef-fce7-44e7-8aaa-06ee1950cd18 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:49:48.846730 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cbf2ef-fce7-44e7-8aaa-06ee1950cd18 None admin] PATH_INFO: `/v3/users` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:49:48.847442 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17cbf2ef-fce7-44e7-8aaa-06ee1950cd18 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:49:48.847743 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17cbf2ef-fce7-44e7-8aaa-06ee1950cd18 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:49:48.849157 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17cbf2ef-fce7-44e7-8aaa-06ee1950cd18 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:49:48.857780 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:49:48.866759 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 634/1268] 10.222.0.20 () {56 vars in 1143 bytes} [Tue Aug 30 15:49:48 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:49:48.874348 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54b541b-23d4-45a7-bf0b-e882c86d165d None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:49:48.974982 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54b541b-23d4-45a7-bf0b-e882c86d165d None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:49:49.023247 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4613dd5-efca-4634-bdf9-6b3fd6f3464d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:49:49.024511 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4613dd5-efca-4634-bdf9-6b3fd6f3464d None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:49:49.024511 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4613dd5-efca-4634-bdf9-6b3fd6f3464d None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:49:49.024511 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4613dd5-efca-4634-bdf9-6b3fd6f3464d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:49:49.035477 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4613dd5-efca-4634-bdf9-6b3fd6f3464d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:49:49.036929 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4613dd5-efca-4634-bdf9-6b3fd6f3464d None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:49:49.051871 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application [None req-c4613dd5-efca-4634-bdf9-6b3fd6f3464d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 15:49:49.051871 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:49:49.051871 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:49:49.051871 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:49:49.051871 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:49:49.051871 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:49:49.051871 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:49:49.051871 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:49:49.051871 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:49:49.051871 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:49:49.051871 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:49:49.051871 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:49:49.051871 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 15:49:49.051871 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 15:49:49.051871 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 15:49:49.051871 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 15:49:49.052899 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:49:49.052899 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:49:49.052899 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:49:49.052899 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:49:49.052899 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:49:49.052899 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:49:49.052899 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:49:49.052899 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application with Lock( Aug 30 15:49:49.052899 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:49:49.052899 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:49:49.052899 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:49:49.052899 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:49:49.052899 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:49:49.052899 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:49:49.052899 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:49:49.052899 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:49:49.052899 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 15:49:49.054022 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 15:49:49.054022 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 15:49:49.054022 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Aug 30 15:49:49.054022 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 635/1269] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 30 15:49:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 15:49:49.059187 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cbf2ef-fce7-44e7-8aaa-06ee1950cd18 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:49:49.168392 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cbf2ef-fce7-44e7-8aaa-06ee1950cd18 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:49:49.204876 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893191e6-b3b3-4c31-b92f-c72490894755 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:49:49.205497 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893191e6-b3b3-4c31-b92f-c72490894755 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:49:49.205774 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893191e6-b3b3-4c31-b92f-c72490894755 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:49:49.206044 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893191e6-b3b3-4c31-b92f-c72490894755 None admin] PATH_INFO: `/v3/domains` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:49:49.206682 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893191e6-b3b3-4c31-b92f-c72490894755 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:49:49.206978 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893191e6-b3b3-4c31-b92f-c72490894755 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:49:49.208350 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893191e6-b3b3-4c31-b92f-c72490894755 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:49:49.219708 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 635/1270] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Aug 30 15:49:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 15:49:49.228057 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4613dd5-efca-4634-bdf9-6b3fd6f3464d None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:49:49.260789 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:49:49.330953 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4613dd5-efca-4634-bdf9-6b3fd6f3464d None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:49:49.368137 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2cd13a-dd0e-4122-a57c-eff2b9100bc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:49:49.368757 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2cd13a-dd0e-4122-a57c-eff2b9100bc6 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:49:49.369031 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2cd13a-dd0e-4122-a57c-eff2b9100bc6 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:49:49.369327 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2cd13a-dd0e-4122-a57c-eff2b9100bc6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:49:49.376797 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2cd13a-dd0e-4122-a57c-eff2b9100bc6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:49:49.378282 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2cd13a-dd0e-4122-a57c-eff2b9100bc6 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:49:49.390102 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application [None req-8d2cd13a-dd0e-4122-a57c-eff2b9100bc6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 15:49:49.390102 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:49:49.390102 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:49:49.390102 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:49:49.390102 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:49:49.390102 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:49:49.390102 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:49:49.390102 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:49:49.390102 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:49:49.390102 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:49:49.390102 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:49:49.390102 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:49:49.390102 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 15:49:49.390102 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 15:49:49.390102 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 15:49:49.391160 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 15:49:49.391160 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:49:49.391160 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:49:49.391160 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:49:49.391160 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:49:49.391160 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:49:49.391160 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:49:49.391160 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:49:49.391160 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application with Lock( Aug 30 15:49:49.391160 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:49:49.391160 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:49:49.391160 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:49:49.391160 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:49:49.391160 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:49:49.391160 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:49:49.391160 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:49:49.391160 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:49:49.392273 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 15:49:49.392273 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 15:49:49.392273 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 15:49:49.392273 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 15:49:49.392273 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 15:49:49.392273 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 15:49:49.392273 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 15:49:49.392273 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Aug 30 15:49:49.392273 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 636/1271] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Aug 30 15:49:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 15:49:49.397618 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893191e6-b3b3-4c31-b92f-c72490894755 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:49:49.456901 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:49:49.504487 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893191e6-b3b3-4c31-b92f-c72490894755 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:49:49.547378 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020db480-a5df-453a-af43-fc4ac47df69b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:49:49.548017 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020db480-a5df-453a-af43-fc4ac47df69b None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:49:49.548224 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020db480-a5df-453a-af43-fc4ac47df69b None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:49:49.548566 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020db480-a5df-453a-af43-fc4ac47df69b None admin] PATH_INFO: `/v3/projects` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:49:49.549644 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020db480-a5df-453a-af43-fc4ac47df69b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:49:49.550172 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020db480-a5df-453a-af43-fc4ac47df69b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:49:49.552434 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020db480-a5df-453a-af43-fc4ac47df69b None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:49:49.567664 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 636/1272] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Aug 30 15:49:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:49:49.577660 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2cd13a-dd0e-4122-a57c-eff2b9100bc6 None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:49:49.686969 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:49:49.696718 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2cd13a-dd0e-4122-a57c-eff2b9100bc6 None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:49:49.733169 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725d5bf7-86b7-43ff-98a2-dc7acbcaf867 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:49:49.733850 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725d5bf7-86b7-43ff-98a2-dc7acbcaf867 None admin] REQUEST_METHOD: `PUT` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:49:49.734126 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725d5bf7-86b7-43ff-98a2-dc7acbcaf867 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:49:49.734394 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725d5bf7-86b7-43ff-98a2-dc7acbcaf867 None admin] PATH_INFO: `/v3/projects/854fe71b49224dff9a4b1766a20ab266/users/bc12c6d31eab4a17a160250adc2380c8/roles/f265386950a641ac8413b53eda4ae34d` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:49:49.736552 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725d5bf7-86b7-43ff-98a2-dc7acbcaf867 None admin] RBAC: Authorizing `identity:create_grant(project_id=854fe71b49224dff9a4b1766a20ab266, user_id=bc12c6d31eab4a17a160250adc2380c8, role_id=f265386950a641ac8413b53eda4ae34d)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:49:49.738032 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725d5bf7-86b7-43ff-98a2-dc7acbcaf867 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:49:49.756462 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 637/1273] 10.222.0.20 () {58 vars in 1439 bytes} [Tue Aug 30 15:49:49 2022] PUT /identity/v3/projects/854fe71b49224dff9a4b1766a20ab266/users/bc12c6d31eab4a17a160250adc2380c8/roles/f265386950a641ac8413b53eda4ae34d => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:49:51.630282 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65be554f-cb46-42ea-947d-489199ebf723 None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:49:51.630834 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65be554f-cb46-42ea-947d-489199ebf723 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:49:51.630834 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65be554f-cb46-42ea-947d-489199ebf723 None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:49:51.632181 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 637/1274] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 30 15:49:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 15:49:51.638502 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820366f7-24ec-4565-af6b-7dfdb491f9d0 None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:49:51.639200 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820366f7-24ec-4565-af6b-7dfdb491f9d0 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:49:51.639338 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820366f7-24ec-4565-af6b-7dfdb491f9d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:49:51.661500 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-820366f7-24ec-4565-af6b-7dfdb491f9d0 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:49:51.662692 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-820366f7-24ec-4565-af6b-7dfdb491f9d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:49:51.744149 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 638/1275] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 30 15:49:51 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 15:49:52.006176 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65be554f-cb46-42ea-947d-489199ebf723 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:49:52.117804 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65be554f-cb46-42ea-947d-489199ebf723 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:49:52.137570 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:49:52.156911 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e49700-eb10-4e3c-9bab-a28999b7a46f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:49:52.157579 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e49700-eb10-4e3c-9bab-a28999b7a46f None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:49:52.157865 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e49700-eb10-4e3c-9bab-a28999b7a46f None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:49:52.158131 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e49700-eb10-4e3c-9bab-a28999b7a46f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:49:52.168650 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e49700-eb10-4e3c-9bab-a28999b7a46f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 15:49:52.173584 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-b5e49700-eb10-4e3c-9bab-a28999b7a46f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 15:49:52.173584 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:49:52.173584 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:49:52.173584 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:49:52.173584 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:49:52.173584 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:49:52.173584 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:49:52.173584 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:49:52.173584 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:49:52.173584 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:49:52.173584 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:49:52.173584 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:49:52.173584 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 15:49:52.173584 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 15:49:52.173584 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 15:49:52.173584 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 15:49:52.174697 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 15:49:52.174697 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 15:49:52.174697 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 15:49:52.174697 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 15:49:52.174697 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:49:52.174697 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:49:52.174697 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:49:52.174697 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:49:52.174697 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:49:52.174697 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:49:52.174697 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:49:52.174697 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Aug 30 15:49:52.174697 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:49:52.174697 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:49:52.174697 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:49:52.174697 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:49:52.176089 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:49:52.176089 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:49:52.176089 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:49:52.176089 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:49:52.176089 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 15:49:52.176089 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 15:49:52.176089 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 15:49:52.176089 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 15:49:52.176089 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 15:49:52.176089 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 15:49:52.176089 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 15:49:52.176089 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Aug 30 15:49:52.176089 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 638/1276] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Aug 30 15:49:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 15:49:52.183729 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820366f7-24ec-4565-af6b-7dfdb491f9d0 None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:49:52.266562 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:49:52.288624 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820366f7-24ec-4565-af6b-7dfdb491f9d0 None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:49:52.325138 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1548e02f-b182-416c-ae78-f578ed6399a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:49:52.325759 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1548e02f-b182-416c-ae78-f578ed6399a6 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:49:52.326032 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1548e02f-b182-416c-ae78-f578ed6399a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:49:52.326319 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1548e02f-b182-416c-ae78-f578ed6399a6 None admin] PATH_INFO: `/v3/roles` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:49:52.327047 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1548e02f-b182-416c-ae78-f578ed6399a6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81512) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:49:52.327340 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1548e02f-b182-416c-ae78-f578ed6399a6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:49:52.328725 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1548e02f-b182-416c-ae78-f578ed6399a6 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:49:52.336496 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 639/1277] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Aug 30 15:49:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:49:52.342539 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e49700-eb10-4e3c-9bab-a28999b7a46f None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:49:52.448525 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e49700-eb10-4e3c-9bab-a28999b7a46f None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:49:52.485938 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3fb090-8dd9-4659-9dd4-53c5a3c3c91c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:49:52.486555 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3fb090-8dd9-4659-9dd4-53c5a3c3c91c None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:49:52.486825 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3fb090-8dd9-4659-9dd4-53c5a3c3c91c None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:49:52.487089 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3fb090-8dd9-4659-9dd4-53c5a3c3c91c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:49:52.497953 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3fb090-8dd9-4659-9dd4-53c5a3c3c91c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:49:52.499347 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3fb090-8dd9-4659-9dd4-53c5a3c3c91c None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:49:52.514779 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-dc3fb090-8dd9-4659-9dd4-53c5a3c3c91c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 15:49:52.514779 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:49:52.514779 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:49:52.514779 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:49:52.514779 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:49:52.514779 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:49:52.514779 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:49:52.514779 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:49:52.514779 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:49:52.514779 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:49:52.514779 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:49:52.514779 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:49:52.514779 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 15:49:52.514779 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 15:49:52.514779 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 15:49:52.514779 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 15:49:52.515848 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:49:52.515848 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:49:52.515848 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:49:52.515848 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:49:52.515848 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:49:52.515848 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:49:52.515848 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:49:52.515848 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Aug 30 15:49:52.515848 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:49:52.515848 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:49:52.515848 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:49:52.515848 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:49:52.515848 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:49:52.515848 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:49:52.515848 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:49:52.515848 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:49:52.515848 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 15:49:52.516962 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 15:49:52.516962 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 15:49:52.516962 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Aug 30 15:49:52.520635 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 639/1278] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 30 15:49:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 15:49:52.521881 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1548e02f-b182-416c-ae78-f578ed6399a6 None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:49:52.622350 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1548e02f-b182-416c-ae78-f578ed6399a6 None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:49:52.660214 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b86cbe-70d4-4ddd-ba48-85ceb29a5f73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:49:52.660934 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b86cbe-70d4-4ddd-ba48-85ceb29a5f73 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:49:52.661039 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b86cbe-70d4-4ddd-ba48-85ceb29a5f73 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:49:52.661352 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b86cbe-70d4-4ddd-ba48-85ceb29a5f73 None admin] PATH_INFO: `/v3/domains` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:49:52.661989 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b86cbe-70d4-4ddd-ba48-85ceb29a5f73 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81512) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:49:52.662285 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b86cbe-70d4-4ddd-ba48-85ceb29a5f73 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:49:52.663648 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b86cbe-70d4-4ddd-ba48-85ceb29a5f73 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:49:52.676486 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 640/1279] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Aug 30 15:49:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:49:52.682574 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3fb090-8dd9-4659-9dd4-53c5a3c3c91c None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:49:52.715826 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:49:52.786600 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3fb090-8dd9-4659-9dd4-53c5a3c3c91c None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:49:52.822335 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64631d63-da3e-4447-bd20-c41edbb81b21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:49:52.822947 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64631d63-da3e-4447-bd20-c41edbb81b21 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:49:52.823216 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64631d63-da3e-4447-bd20-c41edbb81b21 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:49:52.823478 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64631d63-da3e-4447-bd20-c41edbb81b21 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:49:52.831017 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64631d63-da3e-4447-bd20-c41edbb81b21 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:49:52.832451 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64631d63-da3e-4447-bd20-c41edbb81b21 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:49:52.843703 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-64631d63-da3e-4447-bd20-c41edbb81b21 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 30 15:49:52.843703 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:49:52.843703 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:49:52.843703 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:49:52.843703 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:49:52.843703 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:49:52.843703 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:49:52.843703 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:49:52.843703 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:49:52.843703 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:49:52.843703 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:49:52.843703 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:49:52.843703 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 15:49:52.843703 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 15:49:52.843703 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 15:49:52.843703 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 15:49:52.844727 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:49:52.844727 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:49:52.844727 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 15:49:52.844727 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 15:49:52.844727 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 15:49:52.844727 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 15:49:52.844727 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:49:52.844727 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:49:52.844727 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:49:52.844727 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:49:52.844727 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:49:52.844727 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Aug 30 15:49:52.844727 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:49:52.844727 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:49:52.844727 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:49:52.844727 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:49:52.844727 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:49:52.844727 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:49:52.845882 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:49:52.845882 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:49:52.845882 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 15:49:52.845882 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 15:49:52.845882 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 15:49:52.845882 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 15:49:52.845882 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 15:49:52.845882 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 15:49:52.845882 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 30 15:49:52.845882 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Aug 30 15:49:52.845882 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 640/1280] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 30 15:49:52 2022] GET /identity/v3/users/placement => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 15:49:52.850712 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b86cbe-70d4-4ddd-ba48-85ceb29a5f73 None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:49:52.854712 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:49:52.953741 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b86cbe-70d4-4ddd-ba48-85ceb29a5f73 None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:49:52.989237 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957fe7e5-139a-414b-8ba1-5a4c55ee6a2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:49:52.989852 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957fe7e5-139a-414b-8ba1-5a4c55ee6a2c None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:49:52.990555 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957fe7e5-139a-414b-8ba1-5a4c55ee6a2c None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:49:52.990827 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957fe7e5-139a-414b-8ba1-5a4c55ee6a2c None admin] PATH_INFO: `/v3/users` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:49:52.991595 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957fe7e5-139a-414b-8ba1-5a4c55ee6a2c None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81512) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:49:52.991883 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957fe7e5-139a-414b-8ba1-5a4c55ee6a2c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:49:52.993450 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957fe7e5-139a-414b-8ba1-5a4c55ee6a2c None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:49:53.012746 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 641/1281] 10.222.0.20 () {56 vars in 1143 bytes} [Tue Aug 30 15:49:52 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:49:53.019330 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64631d63-da3e-4447-bd20-c41edbb81b21 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:49:53.110836 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:49:53.121176 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64631d63-da3e-4447-bd20-c41edbb81b21 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:49:53.155663 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c5118d-3762-4835-bd5d-ec68e9decc2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:49:53.156346 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c5118d-3762-4835-bd5d-ec68e9decc2e None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:49:53.156619 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c5118d-3762-4835-bd5d-ec68e9decc2e None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:49:53.156884 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c5118d-3762-4835-bd5d-ec68e9decc2e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:49:53.222977 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c5118d-3762-4835-bd5d-ec68e9decc2e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:49:53.224418 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c5118d-3762-4835-bd5d-ec68e9decc2e None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:49:53.314092 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:49:53.326135 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 641/1282] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Aug 30 15:49:53 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 30 15:49:53.335758 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957fe7e5-139a-414b-8ba1-5a4c55ee6a2c None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:49:53.402079 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:49:53.442537 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957fe7e5-139a-414b-8ba1-5a4c55ee6a2c None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:49:53.478233 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9db803d-f451-49da-8038-0a73fe1b5c5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:49:53.478844 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9db803d-f451-49da-8038-0a73fe1b5c5e None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:49:53.479115 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9db803d-f451-49da-8038-0a73fe1b5c5e None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:49:53.479376 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9db803d-f451-49da-8038-0a73fe1b5c5e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:49:53.489721 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9db803d-f451-49da-8038-0a73fe1b5c5e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:49:53.491503 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9db803d-f451-49da-8038-0a73fe1b5c5e None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:49:53.506234 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application [None req-e9db803d-f451-49da-8038-0a73fe1b5c5e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 15:49:53.506234 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:49:53.506234 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:49:53.506234 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:49:53.506234 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:49:53.506234 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:49:53.506234 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:49:53.506234 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:49:53.506234 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:49:53.506234 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:49:53.506234 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:49:53.506234 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:49:53.506234 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 15:49:53.506234 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 15:49:53.506234 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 15:49:53.506234 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 15:49:53.507319 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:49:53.507319 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:49:53.507319 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:49:53.507319 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:49:53.507319 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:49:53.507319 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:49:53.507319 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:49:53.507319 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application with Lock( Aug 30 15:49:53.507319 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:49:53.507319 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:49:53.507319 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:49:53.507319 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:49:53.507319 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:49:53.507319 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:49:53.507319 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:49:53.507319 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:49:53.507319 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 15:49:53.508246 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 15:49:53.508246 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 15:49:53.508246 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Aug 30 15:49:53.508246 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 642/1283] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 30 15:49:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 15:49:53.513405 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c5118d-3762-4835-bd5d-ec68e9decc2e None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:49:53.615505 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c5118d-3762-4835-bd5d-ec68e9decc2e None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:49:53.653207 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d8cc68-bc34-481e-bd75-5b7c849e1a32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:49:53.653819 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d8cc68-bc34-481e-bd75-5b7c849e1a32 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:49:53.654100 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d8cc68-bc34-481e-bd75-5b7c849e1a32 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:49:53.654390 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d8cc68-bc34-481e-bd75-5b7c849e1a32 None admin] PATH_INFO: `/v3/domains` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:49:53.655041 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d8cc68-bc34-481e-bd75-5b7c849e1a32 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:49:53.655343 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d8cc68-bc34-481e-bd75-5b7c849e1a32 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:49:53.656725 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d8cc68-bc34-481e-bd75-5b7c849e1a32 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:49:53.668426 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 642/1284] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Aug 30 15:49:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:49:53.674363 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9db803d-f451-49da-8038-0a73fe1b5c5e None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:49:53.778534 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9db803d-f451-49da-8038-0a73fe1b5c5e None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:49:53.792452 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:49:53.812666 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ed470e-d0ba-4534-b3f3-3fd301eaccba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:49:53.813320 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ed470e-d0ba-4534-b3f3-3fd301eaccba None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:49:53.813611 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ed470e-d0ba-4534-b3f3-3fd301eaccba None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:49:53.813879 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ed470e-d0ba-4534-b3f3-3fd301eaccba None admin] PATH_INFO: `/v3/projects/service` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:49:53.820502 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ed470e-d0ba-4534-b3f3-3fd301eaccba None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:49:53.821971 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ed470e-d0ba-4534-b3f3-3fd301eaccba None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:49:53.833912 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application [None req-06ed470e-d0ba-4534-b3f3-3fd301eaccba None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 15:49:53.833912 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:49:53.833912 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:49:53.833912 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:49:53.833912 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:49:53.833912 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:49:53.833912 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:49:53.833912 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:49:53.833912 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:49:53.833912 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:49:53.833912 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:49:53.833912 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:49:53.833912 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 15:49:53.833912 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 15:49:53.833912 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 15:49:53.835002 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 15:49:53.835002 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:49:53.835002 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:49:53.835002 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:49:53.835002 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:49:53.835002 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:49:53.835002 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:49:53.835002 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:49:53.835002 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application with Lock( Aug 30 15:49:53.835002 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:49:53.835002 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:49:53.835002 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:49:53.835002 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:49:53.835002 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:49:53.835002 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:49:53.835002 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:49:53.835002 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:49:53.835909 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 15:49:53.835909 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 15:49:53.835909 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 15:49:53.835909 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 15:49:53.835909 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 15:49:53.835909 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 15:49:53.835909 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 15:49:53.835909 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: ERROR keystone.server.flask.application Aug 30 15:49:53.836953 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 643/1285] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Aug 30 15:49:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 15:49:53.841148 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d8cc68-bc34-481e-bd75-5b7c849e1a32 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:49:53.891830 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:49:53.941920 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d8cc68-bc34-481e-bd75-5b7c849e1a32 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:49:53.974225 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e607ebf1-8ae0-4a3b-93cb-58067078deeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:49:53.974830 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e607ebf1-8ae0-4a3b-93cb-58067078deeb None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:49:53.975134 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e607ebf1-8ae0-4a3b-93cb-58067078deeb None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:49:53.975399 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e607ebf1-8ae0-4a3b-93cb-58067078deeb None admin] PATH_INFO: `/v3/projects` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:49:53.976084 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e607ebf1-8ae0-4a3b-93cb-58067078deeb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:49:53.976382 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e607ebf1-8ae0-4a3b-93cb-58067078deeb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:49:53.977842 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e607ebf1-8ae0-4a3b-93cb-58067078deeb None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:49:53.995983 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 643/1286] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Aug 30 15:49:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:49:53.997171 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ed470e-d0ba-4534-b3f3-3fd301eaccba None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:49:54.104123 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ed470e-d0ba-4534-b3f3-3fd301eaccba None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:49:54.151739 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840f4018-9f21-4e2c-ab06-94f79a28c6e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:49:54.152617 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840f4018-9f21-4e2c-ab06-94f79a28c6e1 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:49:54.153022 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840f4018-9f21-4e2c-ab06-94f79a28c6e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:49:54.153685 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840f4018-9f21-4e2c-ab06-94f79a28c6e1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:49:54.154612 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840f4018-9f21-4e2c-ab06-94f79a28c6e1 None admin] RBAC: Adding query filter params (role.id=f265386950a641ac8413b53eda4ae34d, scope.project.id=854fe71b49224dff9a4b1766a20ab266, user.id=bc12c6d31eab4a17a160250adc2380c8) {{(pid=81512) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:49:54.154918 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840f4018-9f21-4e2c-ab06-94f79a28c6e1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:49:54.156354 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840f4018-9f21-4e2c-ab06-94f79a28c6e1 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:49:54.162429 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 644/1287] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Aug 30 15:49:53 2022] GET /identity/v3/role_assignments?user.id=bc12c6d31eab4a17a160250adc2380c8&scope.project.id=854fe71b49224dff9a4b1766a20ab266&role.id=f265386950a641ac8413b53eda4ae34d => generated 608 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:49:56.067100 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0754ce73-5c9f-4540-a3ae-26de60b20921 None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:49:56.067952 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0754ce73-5c9f-4540-a3ae-26de60b20921 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:49:56.067952 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0754ce73-5c9f-4540-a3ae-26de60b20921 None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:49:56.069977 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 644/1288] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 30 15:49:56 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 15:49:56.078592 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fd8c92-9ab0-4a44-ab66-1eeaf108adfc None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:49:56.079019 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fd8c92-9ab0-4a44-ab66-1eeaf108adfc None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:49:56.079415 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fd8c92-9ab0-4a44-ab66-1eeaf108adfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:49:56.102100 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-f1fd8c92-9ab0-4a44-ab66-1eeaf108adfc None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:49:56.103304 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-f1fd8c92-9ab0-4a44-ab66-1eeaf108adfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:49:56.134825 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:49:56.174528 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 645/1289] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 30 15:49:56 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 15:49:56.443354 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0754ce73-5c9f-4540-a3ae-26de60b20921 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:49:56.552612 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0754ce73-5c9f-4540-a3ae-26de60b20921 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:49:56.563181 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:49:56.588998 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c240c143-20d5-4c32-a85b-c3c4713fd782 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:49:56.589640 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c240c143-20d5-4c32-a85b-c3c4713fd782 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:49:56.589908 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c240c143-20d5-4c32-a85b-c3c4713fd782 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:49:56.590171 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c240c143-20d5-4c32-a85b-c3c4713fd782 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:49:56.590751 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c240c143-20d5-4c32-a85b-c3c4713fd782 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:49:56.592301 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c240c143-20d5-4c32-a85b-c3c4713fd782 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:49:56.601498 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-c240c143-20d5-4c32-a85b-c3c4713fd782 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 30 15:49:56.601498 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:49:56.601498 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 15:49:56.601498 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 15:49:56.601498 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 15:49:56.601498 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 15:49:56.601498 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 15:49:56.601498 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 15:49:56.601498 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 30 15:49:56.601498 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Aug 30 15:49:56.601498 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 15:49:56.601498 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Aug 30 15:49:56.601498 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:49:56.601498 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:49:56.601498 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:49:56.601498 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:49:56.601498 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:49:56.602761 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:49:56.602761 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:49:56.602761 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:49:56.602761 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:49:56.602761 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:49:56.602761 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:49:56.602761 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 15:49:56.602761 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 15:49:56.602761 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 15:49:56.602761 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 15:49:56.602761 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:49:56.602761 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:49:56.602761 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:49:56.602761 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:49:56.602761 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:49:56.602761 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:49:56.604164 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:49:56.604164 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Aug 30 15:49:56.604164 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:49:56.604164 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:49:56.604164 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:49:56.604164 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:49:56.604164 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:49:56.604164 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:49:56.604164 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:49:56.604164 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:49:56.604164 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 15:49:56.604164 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 15:49:56.604164 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 30 15:49:56.604164 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Aug 30 15:49:56.604164 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 645/1290] 10.222.0.20 () {56 vars in 1117 bytes} [Tue Aug 30 15:49:56 2022] GET /identity/v3/services/placement => generated 90 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 15:49:56.612217 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fd8c92-9ab0-4a44-ab66-1eeaf108adfc None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:49:56.715278 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fd8c92-9ab0-4a44-ab66-1eeaf108adfc None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:49:56.752829 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4008f597-4572-4087-999e-5050aa7b0ecc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:49:56.753458 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4008f597-4572-4087-999e-5050aa7b0ecc None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:49:56.753735 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4008f597-4572-4087-999e-5050aa7b0ecc None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:49:56.754004 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4008f597-4572-4087-999e-5050aa7b0ecc None admin] PATH_INFO: `/v3/services` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:49:56.754631 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4008f597-4572-4087-999e-5050aa7b0ecc None admin] RBAC: Adding query filter params (name=placement) {{(pid=81512) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:49:56.754932 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4008f597-4572-4087-999e-5050aa7b0ecc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:49:56.756312 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4008f597-4572-4087-999e-5050aa7b0ecc None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:49:56.761714 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 646/1291] 10.222.0.20 () {56 vars in 1116 bytes} [Tue Aug 30 15:49:56 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:49:56.767942 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c240c143-20d5-4c32-a85b-c3c4713fd782 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:49:56.879828 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c240c143-20d5-4c32-a85b-c3c4713fd782 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:49:56.913975 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00932050-d9e8-4b27-8ffb-e9bee870fc40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:49:56.914563 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00932050-d9e8-4b27-8ffb-e9bee870fc40 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:49:56.914839 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00932050-d9e8-4b27-8ffb-e9bee870fc40 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:49:56.915109 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00932050-d9e8-4b27-8ffb-e9bee870fc40 None admin] PATH_INFO: `/v3/services` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:49:56.915734 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00932050-d9e8-4b27-8ffb-e9bee870fc40 None admin] RBAC: Adding query filter params (type=placement) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:49:56.916037 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00932050-d9e8-4b27-8ffb-e9bee870fc40 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:49:56.917433 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00932050-d9e8-4b27-8ffb-e9bee870fc40 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:49:56.922047 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 646/1292] 10.222.0.20 () {56 vars in 1116 bytes} [Tue Aug 30 15:49:56 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:49:58.814874 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d481032-b997-4d7f-be50-e297ef096799 None None] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:49:58.815434 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d481032-b997-4d7f-be50-e297ef096799 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:49:58.815434 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d481032-b997-4d7f-be50-e297ef096799 None None] PATH_INFO: `/` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:49:58.816762 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 647/1293] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 30 15:49:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 15:49:58.823247 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c7698f-c60b-400f-ab04-8e52825c6c42 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:49:58.824008 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c7698f-c60b-400f-ab04-8e52825c6c42 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:49:58.824133 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c7698f-c60b-400f-ab04-8e52825c6c42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:49:58.845442 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-60c7698f-c60b-400f-ab04-8e52825c6c42 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:49:58.846636 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-60c7698f-c60b-400f-ab04-8e52825c6c42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:49:58.880035 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:49:58.922742 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 647/1294] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 30 15:49:58 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 15:49:59.191829 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d481032-b997-4d7f-be50-e297ef096799 None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:49:59.227206 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:49:59.303560 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d481032-b997-4d7f-be50-e297ef096799 None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:49:59.339681 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669bd11f-22b6-40b7-b392-184293e844e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:49:59.340270 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669bd11f-22b6-40b7-b392-184293e844e9 None admin] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:49:59.340538 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669bd11f-22b6-40b7-b392-184293e844e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:49:59.340822 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669bd11f-22b6-40b7-b392-184293e844e9 None admin] PATH_INFO: `/v3/services` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:49:59.341521 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669bd11f-22b6-40b7-b392-184293e844e9 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:49:59.343004 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669bd11f-22b6-40b7-b392-184293e844e9 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:49:59.360570 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 648/1295] 10.222.0.20 () {60 vars in 1141 bytes} [Tue Aug 30 15:49:59 2022] POST /identity/v3/services => generated 246 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 15:50:01.254712 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc681e16-94eb-41bd-b199-d3b30f082ffb None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:50:01.255295 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc681e16-94eb-41bd-b199-d3b30f082ffb None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:50:01.255295 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc681e16-94eb-41bd-b199-d3b30f082ffb None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:50:01.256587 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 648/1296] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 30 15:50:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 15:50:01.262998 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e29492-c6bd-4f3b-b03c-8605910584d6 None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:50:01.263438 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e29492-c6bd-4f3b-b03c-8605910584d6 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:50:01.263605 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e29492-c6bd-4f3b-b03c-8605910584d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:50:01.284396 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-68e29492-c6bd-4f3b-b03c-8605910584d6 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:50:01.285614 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-68e29492-c6bd-4f3b-b03c-8605910584d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:50:01.360573 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 649/1297] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 30 15:50:01 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 15:50:01.630513 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc681e16-94eb-41bd-b199-d3b30f082ffb None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:50:01.745487 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc681e16-94eb-41bd-b199-d3b30f082ffb None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:50:01.779774 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1899b8-ff83-45c7-964d-aa26d283474f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:50:01.780401 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1899b8-ff83-45c7-964d-aa26d283474f None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:50:01.780676 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1899b8-ff83-45c7-964d-aa26d283474f None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:50:01.780950 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1899b8-ff83-45c7-964d-aa26d283474f None admin] PATH_INFO: `/v3/services/placement` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:50:01.781557 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1899b8-ff83-45c7-964d-aa26d283474f None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:50:01.782932 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1899b8-ff83-45c7-964d-aa26d283474f None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:50:01.791804 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-bc1899b8-ff83-45c7-964d-aa26d283474f None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 30 15:50:01.791804 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:50:01.791804 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 15:50:01.791804 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 15:50:01.791804 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 15:50:01.791804 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 15:50:01.791804 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 15:50:01.791804 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 15:50:01.791804 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 30 15:50:01.791804 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Aug 30 15:50:01.791804 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 15:50:01.791804 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Aug 30 15:50:01.791804 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:50:01.791804 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:50:01.791804 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:50:01.791804 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:50:01.791804 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:50:01.792973 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:50:01.792973 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:50:01.792973 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:50:01.792973 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:50:01.792973 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:50:01.792973 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:50:01.792973 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 15:50:01.792973 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 15:50:01.792973 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 15:50:01.792973 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 15:50:01.792973 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:50:01.792973 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:50:01.792973 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:50:01.792973 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:50:01.792973 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:50:01.792973 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:50:01.793974 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:50:01.793974 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Aug 30 15:50:01.793974 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:50:01.793974 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:50:01.793974 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:50:01.793974 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:50:01.793974 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:50:01.793974 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:50:01.793974 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:50:01.793974 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:50:01.793974 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 15:50:01.793974 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 15:50:01.793974 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 30 15:50:01.793974 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Aug 30 15:50:01.793974 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 649/1298] 10.222.0.20 () {56 vars in 1117 bytes} [Tue Aug 30 15:50:01 2022] GET /identity/v3/services/placement => generated 90 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 15:50:01.799229 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e29492-c6bd-4f3b-b03c-8605910584d6 None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:50:01.819794 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:50:01.903225 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e29492-c6bd-4f3b-b03c-8605910584d6 None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:50:01.938712 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022a74af-98dc-47ef-a644-17085c086805 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:50:01.939322 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022a74af-98dc-47ef-a644-17085c086805 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:50:01.939562 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022a74af-98dc-47ef-a644-17085c086805 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:50:01.939840 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022a74af-98dc-47ef-a644-17085c086805 None admin] PATH_INFO: `/v3/services` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:50:01.940459 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022a74af-98dc-47ef-a644-17085c086805 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81512) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:50:01.940757 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022a74af-98dc-47ef-a644-17085c086805 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:50:01.942165 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022a74af-98dc-47ef-a644-17085c086805 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:50:01.947765 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 650/1299] 10.222.0.20 () {56 vars in 1116 bytes} [Tue Aug 30 15:50:01 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:50:01.954160 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1899b8-ff83-45c7-964d-aa26d283474f None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:50:01.991021 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:50:02.064019 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1899b8-ff83-45c7-964d-aa26d283474f None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:50:02.098833 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35254419-3cee-427a-a03c-a8acfe1ac38f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:50:02.099427 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35254419-3cee-427a-a03c-a8acfe1ac38f None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:50:02.099701 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35254419-3cee-427a-a03c-a8acfe1ac38f None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:50:02.099964 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35254419-3cee-427a-a03c-a8acfe1ac38f None admin] PATH_INFO: `/v3/endpoints` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:50:02.100662 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35254419-3cee-427a-a03c-a8acfe1ac38f None admin] RBAC: Adding query filter params (interface=public, service_id=515b9995aa5446d69f015c2bd3c925e0, region_id=RegionOne) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:50:02.100969 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35254419-3cee-427a-a03c-a8acfe1ac38f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:50:02.102374 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35254419-3cee-427a-a03c-a8acfe1ac38f None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:50:02.109771 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 650/1300] 10.222.0.20 () {56 vars in 1251 bytes} [Tue Aug 30 15:50:01 2022] GET /identity/v3/endpoints?service_id=515b9995aa5446d69f015c2bd3c925e0&interface=public®ion_id=RegionOne => generated 194 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 15:50:02.116242 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022a74af-98dc-47ef-a644-17085c086805 None admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:50:02.197280 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:50:02.223313 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022a74af-98dc-47ef-a644-17085c086805 None admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:50:02.257234 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a52e091-20b0-4933-80e8-b557cf322b72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:50:02.257848 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a52e091-20b0-4933-80e8-b557cf322b72 None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:50:02.258126 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a52e091-20b0-4933-80e8-b557cf322b72 None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:50:02.258396 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a52e091-20b0-4933-80e8-b557cf322b72 None admin] PATH_INFO: `/v3/services` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:50:02.258995 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a52e091-20b0-4933-80e8-b557cf322b72 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:50:02.260384 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a52e091-20b0-4933-80e8-b557cf322b72 None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:50:02.267037 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 651/1301] 10.222.0.20 () {56 vars in 1087 bytes} [Tue Aug 30 15:50:02 2022] GET /identity/v3/services => generated 2191 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 15:50:04.144848 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536607f8-c962-49aa-a66a-cc66ef4fcb1b None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:50:04.145425 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536607f8-c962-49aa-a66a-cc66ef4fcb1b None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:50:04.145425 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536607f8-c962-49aa-a66a-cc66ef4fcb1b None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:50:04.146778 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 651/1302] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 30 15:50:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 15:50:04.153023 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331aede2-94c7-4f4e-83d8-71b880606838 None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:50:04.153797 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331aede2-94c7-4f4e-83d8-71b880606838 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:50:04.153959 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331aede2-94c7-4f4e-83d8-71b880606838 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:50:04.179437 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-331aede2-94c7-4f4e-83d8-71b880606838 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:50:04.180640 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-331aede2-94c7-4f4e-83d8-71b880606838 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:50:04.256383 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 652/1303] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 30 15:50:04 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 15:50:04.525073 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536607f8-c962-49aa-a66a-cc66ef4fcb1b None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:50:04.636815 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:50:04.645725 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536607f8-c962-49aa-a66a-cc66ef4fcb1b None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:50:04.684964 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf103f0-f117-4d57-9098-789b0ecb6f6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:50:04.685796 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf103f0-f117-4d57-9098-789b0ecb6f6c None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:50:04.686070 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf103f0-f117-4d57-9098-789b0ecb6f6c None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:50:04.686332 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf103f0-f117-4d57-9098-789b0ecb6f6c None admin] PATH_INFO: `/v3/services/placement` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:50:04.686906 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf103f0-f117-4d57-9098-789b0ecb6f6c None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:50:04.688273 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf103f0-f117-4d57-9098-789b0ecb6f6c None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:50:04.697840 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-dbf103f0-f117-4d57-9098-789b0ecb6f6c None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 30 15:50:04.697840 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:50:04.697840 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 15:50:04.697840 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 15:50:04.697840 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 15:50:04.697840 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 15:50:04.697840 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 15:50:04.697840 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 15:50:04.697840 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 30 15:50:04.697840 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Aug 30 15:50:04.697840 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 15:50:04.697840 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Aug 30 15:50:04.697840 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 15:50:04.697840 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 15:50:04.697840 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 15:50:04.697840 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 15:50:04.697840 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 15:50:04.698941 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 15:50:04.698941 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 15:50:04.698941 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 15:50:04.698941 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 15:50:04.698941 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 15:50:04.698941 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 15:50:04.698941 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 15:50:04.698941 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 15:50:04.698941 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 15:50:04.698941 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 15:50:04.698941 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 15:50:04.698941 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 15:50:04.698941 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 15:50:04.698941 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 15:50:04.698941 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 15:50:04.698941 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 15:50:04.699798 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 15:50:04.699798 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Aug 30 15:50:04.699798 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 15:50:04.699798 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Aug 30 15:50:04.699798 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 15:50:04.699798 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 15:50:04.699798 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 15:50:04.699798 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Aug 30 15:50:04.699798 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 15:50:04.699798 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Aug 30 15:50:04.699798 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 15:50:04.699798 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 15:50:04.699798 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 30 15:50:04.699798 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Aug 30 15:50:04.699798 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 652/1304] 10.222.0.20 () {56 vars in 1117 bytes} [Tue Aug 30 15:50:04 2022] GET /identity/v3/services/placement => generated 90 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 15:50:04.705421 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331aede2-94c7-4f4e-83d8-71b880606838 None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:50:04.786241 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:50:04.813900 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331aede2-94c7-4f4e-83d8-71b880606838 None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:50:04.851011 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9ebace-13fd-47c0-97b8-d30b5f99cc4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:50:04.851610 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9ebace-13fd-47c0-97b8-d30b5f99cc4f None admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:50:04.851881 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9ebace-13fd-47c0-97b8-d30b5f99cc4f None admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:50:04.852149 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9ebace-13fd-47c0-97b8-d30b5f99cc4f None admin] PATH_INFO: `/v3/services` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:50:04.852772 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9ebace-13fd-47c0-97b8-d30b5f99cc4f None admin] RBAC: Adding query filter params (name=placement) {{(pid=81512) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:50:04.853070 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9ebace-13fd-47c0-97b8-d30b5f99cc4f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:50:04.854476 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9ebace-13fd-47c0-97b8-d30b5f99cc4f None admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:50:04.859827 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 653/1305] 10.222.0.20 () {56 vars in 1116 bytes} [Tue Aug 30 15:50:04 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:50:04.866356 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf103f0-f117-4d57-9098-789b0ecb6f6c None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:50:04.971533 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf103f0-f117-4d57-9098-789b0ecb6f6c None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:50:05.006772 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5f6c5d-58fa-45b6-8a64-d82736d704c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:50:05.007378 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5f6c5d-58fa-45b6-8a64-d82736d704c7 None admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:50:05.007656 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5f6c5d-58fa-45b6-8a64-d82736d704c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:50:05.007927 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5f6c5d-58fa-45b6-8a64-d82736d704c7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:50:05.008587 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5f6c5d-58fa-45b6-8a64-d82736d704c7 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:50:05.009988 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5f6c5d-58fa-45b6-8a64-d82736d704c7 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:50:05.029916 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 653/1306] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Aug 30 15:50:04 2022] POST /identity/v3/endpoints => generated 331 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 15:50:14.856979 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=30 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:50:15.025475 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:51:09.056310 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e5ac64-a7b9-493e-80f9-2e8a70d4a226 None None] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:51:09.056916 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e5ac64-a7b9-493e-80f9-2e8a70d4a226 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:51:09.056916 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e5ac64-a7b9-493e-80f9-2e8a70d4a226 None None] PATH_INFO: `/` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:51:09.063603 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ec222d-8187-469d-b795-2be82054251e None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:51:09.064067 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ec222d-8187-469d-b795-2be82054251e None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:51:09.064162 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ec222d-8187-469d-b795-2be82054251e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:51:09.067187 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 654/1307] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 30 15:51:09 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 15:51:09.171418 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-34ec222d-8187-469d-b795-2be82054251e None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:51:09.176173 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-34ec222d-8187-469d-b795-2be82054251e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:51:09.349675 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 654/1308] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 30 15:51:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 287 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 15:51:09.638153 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d9a13b-350d-4bda-a804-7517d8ac777b None None] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:51:09.638490 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d9a13b-350d-4bda-a804-7517d8ac777b None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:51:09.638745 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d9a13b-350d-4bda-a804-7517d8ac777b None None] PATH_INFO: `/` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:51:09.642552 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 655/1309] 10.222.0.20 () {54 vars in 965 bytes} [Tue Aug 30 15:51:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 15:51:09.649162 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b0e453-4ded-45ae-8445-3a63ac72121c None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:51:09.649622 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b0e453-4ded-45ae-8445-3a63ac72121c None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:51:09.649995 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b0e453-4ded-45ae-8445-3a63ac72121c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:51:09.731639 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-36b0e453-4ded-45ae-8445-3a63ac72121c None None] MFA Rules not processed for user `65f4157105f3488f807b7475008ab722`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:51:09.732986 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-36b0e453-4ded-45ae-8445-3a63ac72121c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:51:09.760834 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:51:09.829866 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 655/1310] 10.222.0.20 () {58 vars in 1062 bytes} [Tue Aug 30 15:51:09 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:51:09.840600 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d9a13b-350d-4bda-a804-7517d8ac777b None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:51:09.842776 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-74d9a13b-350d-4bda-a804-7517d8ac777b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:51:09.962186 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d9a13b-350d-4bda-a804-7517d8ac777b None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:51:10.040258 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2433ed-7c83-46ef-8c5c-1f99f46b29c7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f4157105f3488f807b7475008ab722', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:51:10.040846 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2433ed-7c83-46ef-8c5c-1f99f46b29c7 service neutron] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:51:10.041124 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2433ed-7c83-46ef-8c5c-1f99f46b29c7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:51:10.041450 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2433ed-7c83-46ef-8c5c-1f99f46b29c7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:51:10.043121 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-9e2433ed-7c83-46ef-8c5c-1f99f46b29c7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:51:10.101009 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:51:10.159755 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2433ed-7c83-46ef-8c5c-1f99f46b29c7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:51:10.161121 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2433ed-7c83-46ef-8c5c-1f99f46b29c7 service neutron] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:51:10.180390 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 656/1311] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 15:51:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:51:12.368174 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7fb877-f812-477a-a24c-f995af7d377e None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:51:12.368789 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7fb877-f812-477a-a24c-f995af7d377e None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:51:12.368789 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7fb877-f812-477a-a24c-f995af7d377e None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:51:12.370349 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 656/1312] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 30 15:51:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 15:51:12.375723 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841bfbaa-e846-4390-bd64-053b1d936be6 None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:51:12.376172 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841bfbaa-e846-4390-bd64-053b1d936be6 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:51:12.376266 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841bfbaa-e846-4390-bd64-053b1d936be6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:51:12.465511 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-841bfbaa-e846-4390-bd64-053b1d936be6 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:51:12.466991 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-841bfbaa-e846-4390-bd64-053b1d936be6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:51:12.582402 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 657/1313] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 30 15:51:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 15:51:12.880653 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7fb877-f812-477a-a24c-f995af7d377e None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:51:12.882537 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-3b7fb877-f812-477a-a24c-f995af7d377e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:51:13.008656 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7fb877-f812-477a-a24c-f995af7d377e None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:51:13.092656 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:51:13.096085 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aaccd2a-c381-4c98-b2d4-04e81899f2cf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f4157105f3488f807b7475008ab722', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:51:13.096700 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aaccd2a-c381-4c98-b2d4-04e81899f2cf service neutron] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:51:13.096976 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aaccd2a-c381-4c98-b2d4-04e81899f2cf service neutron] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:51:13.097251 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aaccd2a-c381-4c98-b2d4-04e81899f2cf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:51:13.099316 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-1aaccd2a-c381-4c98-b2d4-04e81899f2cf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:51:13.226315 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aaccd2a-c381-4c98-b2d4-04e81899f2cf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:51:13.228673 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aaccd2a-c381-4c98-b2d4-04e81899f2cf service neutron] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:51:13.249994 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 657/1314] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 15:51:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:51:15.498800 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176c4c36-84ef-4785-a3a2-a53bb54872f5 None None] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:51:15.499778 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176c4c36-84ef-4785-a3a2-a53bb54872f5 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:51:15.499778 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176c4c36-84ef-4785-a3a2-a53bb54872f5 None None] PATH_INFO: `/` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:51:15.501878 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 658/1315] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 30 15:51:15 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 15:51:15.508250 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e14d839-e753-4cee-9788-0a83c167a325 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:51:15.508981 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e14d839-e753-4cee-9788-0a83c167a325 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:51:15.509117 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e14d839-e753-4cee-9788-0a83c167a325 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:51:15.585040 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-1e14d839-e753-4cee-9788-0a83c167a325 None None] MFA Rules not processed for user `8249204af55844a180f2d1d09c8b1f59`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:51:15.586296 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-1e14d839-e753-4cee-9788-0a83c167a325 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:51:15.639779 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:51:15.685544 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 658/1316] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 30 15:51:15 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:51:15.978091 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176c4c36-84ef-4785-a3a2-a53bb54872f5 None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:51:15.980218 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-176c4c36-84ef-4785-a3a2-a53bb54872f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:51:15.990802 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:51:16.092540 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176c4c36-84ef-4785-a3a2-a53bb54872f5 None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:51:16.180187 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c8039d-91a8-4972-b0e4-04ca90f3173a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f4157105f3488f807b7475008ab722', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:51:16.180793 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c8039d-91a8-4972-b0e4-04ca90f3173a service neutron] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:51:16.181065 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c8039d-91a8-4972-b0e4-04ca90f3173a service neutron] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:51:16.181365 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c8039d-91a8-4972-b0e4-04ca90f3173a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:51:16.182984 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-95c8039d-91a8-4972-b0e4-04ca90f3173a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:51:16.245880 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:51:16.294444 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c8039d-91a8-4972-b0e4-04ca90f3173a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:51:16.295870 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c8039d-91a8-4972-b0e4-04ca90f3173a service neutron] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:51:16.314415 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 659/1317] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 15:51:15 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:51:18.753135 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea113402-b0ea-429c-b82f-3c79cc4bfb27 None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:51:18.754044 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea113402-b0ea-429c-b82f-3c79cc4bfb27 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:51:18.754044 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea113402-b0ea-429c-b82f-3c79cc4bfb27 None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:51:18.756176 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 659/1318] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 30 15:51:18 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 15:51:18.761667 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f004df-b6d8-47b7-87bb-4c22508a04e6 None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:51:18.762136 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f004df-b6d8-47b7-87bb-4c22508a04e6 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:51:18.762247 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f004df-b6d8-47b7-87bb-4c22508a04e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:51:18.829779 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-d1f004df-b6d8-47b7-87bb-4c22508a04e6 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:51:18.830936 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-d1f004df-b6d8-47b7-87bb-4c22508a04e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:51:18.944149 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 660/1319] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 30 15:51:18 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:51:19.224441 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea113402-b0ea-429c-b82f-3c79cc4bfb27 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:51:19.227015 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-ea113402-b0ea-429c-b82f-3c79cc4bfb27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:51:19.348544 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea113402-b0ea-429c-b82f-3c79cc4bfb27 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:51:19.423342 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269b838a-4922-4a61-bb09-e807364a2ee6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f4157105f3488f807b7475008ab722', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:51:19.423945 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269b838a-4922-4a61-bb09-e807364a2ee6 service neutron] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:51:19.424233 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269b838a-4922-4a61-bb09-e807364a2ee6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:51:19.424486 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269b838a-4922-4a61-bb09-e807364a2ee6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:51:19.426111 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-269b838a-4922-4a61-bb09-e807364a2ee6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:51:19.433731 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:51:19.550210 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269b838a-4922-4a61-bb09-e807364a2ee6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:51:19.551581 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269b838a-4922-4a61-bb09-e807364a2ee6 service neutron] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:51:19.571513 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 660/1320] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 15:51:19 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:51:22.115141 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f74feb8-b807-4d3c-9ec5-dc1bc029df4a None None] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:51:22.115664 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f74feb8-b807-4d3c-9ec5-dc1bc029df4a None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:51:22.115664 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f74feb8-b807-4d3c-9ec5-dc1bc029df4a None None] PATH_INFO: `/` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:51:22.117112 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 661/1321] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 30 15:51:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 15:51:22.122510 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67b0e52-c685-4923-bb39-f9870a38b242 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:51:22.122907 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67b0e52-c685-4923-bb39-f9870a38b242 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:51:22.123059 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67b0e52-c685-4923-bb39-f9870a38b242 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:51:22.192190 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-c67b0e52-c685-4923-bb39-f9870a38b242 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:51:22.193445 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-c67b0e52-c685-4923-bb39-f9870a38b242 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:51:22.254323 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:51:22.296685 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 661/1322] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 30 15:51:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:51:22.579887 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f74feb8-b807-4d3c-9ec5-dc1bc029df4a None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:51:22.581522 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-2f74feb8-b807-4d3c-9ec5-dc1bc029df4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:51:22.602146 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:51:22.696999 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f74feb8-b807-4d3c-9ec5-dc1bc029df4a None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:51:22.777501 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7619d04-a2c7-4642-88c0-1edab34a4e35 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f4157105f3488f807b7475008ab722', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:51:22.781529 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7619d04-a2c7-4642-88c0-1edab34a4e35 service neutron] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:51:22.782751 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7619d04-a2c7-4642-88c0-1edab34a4e35 service neutron] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:51:22.783061 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7619d04-a2c7-4642-88c0-1edab34a4e35 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:51:22.784715 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-d7619d04-a2c7-4642-88c0-1edab34a4e35 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:51:22.861521 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:51:22.900169 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7619d04-a2c7-4642-88c0-1edab34a4e35 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:51:22.902465 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7619d04-a2c7-4642-88c0-1edab34a4e35 service neutron] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:51:22.921103 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 662/1323] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 15:51:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:51:24.878503 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22e9580-986f-454a-928d-37685aa835a5 None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:51:24.879171 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22e9580-986f-454a-928d-37685aa835a5 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:51:24.879171 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22e9580-986f-454a-928d-37685aa835a5 None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:51:24.880246 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 662/1324] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 30 15:51:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 15:51:24.885752 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b8d446-43fd-418b-aad8-a3bb0af8e6fe None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:51:24.886193 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b8d446-43fd-418b-aad8-a3bb0af8e6fe None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:51:24.886322 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b8d446-43fd-418b-aad8-a3bb0af8e6fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:51:24.958161 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-d8b8d446-43fd-418b-aad8-a3bb0af8e6fe None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:51:24.959382 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-d8b8d446-43fd-418b-aad8-a3bb0af8e6fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:51:25.060651 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 663/1325] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 30 15:51:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:51:25.335833 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22e9580-986f-454a-928d-37685aa835a5 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:51:25.337525 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-c22e9580-986f-454a-928d-37685aa835a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:51:25.455007 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22e9580-986f-454a-928d-37685aa835a5 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:51:25.526817 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80b3fa4-2d2f-4f65-be01-e6a07f0a9274 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f4157105f3488f807b7475008ab722', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:51:25.527425 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80b3fa4-2d2f-4f65-be01-e6a07f0a9274 service neutron] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:51:25.527698 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80b3fa4-2d2f-4f65-be01-e6a07f0a9274 service neutron] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:51:25.527960 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80b3fa4-2d2f-4f65-be01-e6a07f0a9274 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:51:25.529573 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-d80b3fa4-2d2f-4f65-be01-e6a07f0a9274 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:51:25.546339 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:51:25.639550 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80b3fa4-2d2f-4f65-be01-e6a07f0a9274 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:51:25.642814 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80b3fa4-2d2f-4f65-be01-e6a07f0a9274 service neutron] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:51:25.659957 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 663/1326] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 15:51:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:51:27.607256 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3cab73-6862-4ce7-a6fc-ee5e8a0dc7b9 None None] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:51:27.607822 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3cab73-6862-4ce7-a6fc-ee5e8a0dc7b9 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:51:27.607984 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3cab73-6862-4ce7-a6fc-ee5e8a0dc7b9 None None] PATH_INFO: `/` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:51:27.609482 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 664/1327] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 30 15:51:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 15:51:27.615261 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce379ab-acd5-4108-b9d3-d1a353c1b24c None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:51:27.615686 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce379ab-acd5-4108-b9d3-d1a353c1b24c None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:51:27.615786 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce379ab-acd5-4108-b9d3-d1a353c1b24c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:51:27.686079 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-bce379ab-acd5-4108-b9d3-d1a353c1b24c None None] MFA Rules not processed for user `8249204af55844a180f2d1d09c8b1f59`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:51:27.687318 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-bce379ab-acd5-4108-b9d3-d1a353c1b24c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:51:27.752501 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:51:27.786185 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 664/1328] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 30 15:51:27 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:51:28.080200 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f745ed29-3cbe-4990-9c8f-b1ae37d85c82 None None] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:51:28.080441 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f745ed29-3cbe-4990-9c8f-b1ae37d85c82 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:51:28.080743 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f745ed29-3cbe-4990-9c8f-b1ae37d85c82 None None] PATH_INFO: `/` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:51:28.082193 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 665/1329] 10.222.0.20 () {54 vars in 965 bytes} [Tue Aug 30 15:51:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 15:51:28.088151 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea617721-8f79-4e9d-af80-d38650f31a42 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:51:28.088656 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea617721-8f79-4e9d-af80-d38650f31a42 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:51:28.088927 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea617721-8f79-4e9d-af80-d38650f31a42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:51:28.169842 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-ea617721-8f79-4e9d-af80-d38650f31a42 None None] MFA Rules not processed for user `65f4157105f3488f807b7475008ab722`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:51:28.171071 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-ea617721-8f79-4e9d-af80-d38650f31a42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:51:28.250293 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 665/1330] 10.222.0.20 () {58 vars in 1062 bytes} [Tue Aug 30 15:51:28 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:51:28.259311 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f745ed29-3cbe-4990-9c8f-b1ae37d85c82 None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:51:28.261008 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-f745ed29-3cbe-4990-9c8f-b1ae37d85c82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:51:28.295608 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:51:28.373254 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f745ed29-3cbe-4990-9c8f-b1ae37d85c82 None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:51:28.451924 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e113ac6a-4b08-437a-a34f-1171a404d167 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f4157105f3488f807b7475008ab722', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:51:28.452576 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e113ac6a-4b08-437a-a34f-1171a404d167 service neutron] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:51:28.452850 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e113ac6a-4b08-437a-a34f-1171a404d167 service neutron] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:51:28.453143 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e113ac6a-4b08-437a-a34f-1171a404d167 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:51:28.454769 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-e113ac6a-4b08-437a-a34f-1171a404d167 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:51:28.551385 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:51:28.590713 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e113ac6a-4b08-437a-a34f-1171a404d167 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:51:28.592150 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e113ac6a-4b08-437a-a34f-1171a404d167 service neutron] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:51:28.617402 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 666/1331] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 15:51:28 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:51:31.124185 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a513f7-ad21-4c74-abe1-93e1ec253ba0 None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:51:31.125168 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a513f7-ad21-4c74-abe1-93e1ec253ba0 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:51:31.125380 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a513f7-ad21-4c74-abe1-93e1ec253ba0 None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:51:31.127484 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 666/1332] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 30 15:51:31 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 15:51:31.133242 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0283db-7136-495f-bec2-6ad011ef9399 None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:51:31.133990 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0283db-7136-495f-bec2-6ad011ef9399 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:51:31.134180 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0283db-7136-495f-bec2-6ad011ef9399 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:51:31.200958 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-3a0283db-7136-495f-bec2-6ad011ef9399 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:51:31.202186 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-3a0283db-7136-495f-bec2-6ad011ef9399 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:51:31.310592 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 667/1333] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 30 15:51:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:51:31.589824 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a513f7-ad21-4c74-abe1-93e1ec253ba0 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:51:31.592346 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-12a513f7-ad21-4c74-abe1-93e1ec253ba0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:51:31.677700 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:51:31.713901 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a513f7-ad21-4c74-abe1-93e1ec253ba0 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:51:31.789815 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afaf125-c314-4dfa-b270-65806ec5d072 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f4157105f3488f807b7475008ab722', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:51:31.790418 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afaf125-c314-4dfa-b270-65806ec5d072 service neutron] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:51:31.790685 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afaf125-c314-4dfa-b270-65806ec5d072 service neutron] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:51:31.790947 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afaf125-c314-4dfa-b270-65806ec5d072 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:51:31.792572 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-4afaf125-c314-4dfa-b270-65806ec5d072 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:51:31.912005 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afaf125-c314-4dfa-b270-65806ec5d072 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:51:31.913488 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afaf125-c314-4dfa-b270-65806ec5d072 service neutron] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:51:31.923675 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:51:31.932863 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 667/1334] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 15:51:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 15:51:33.870275 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501159e7-03ba-4a2e-bc90-720bc4689d0d None None] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:51:33.870869 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501159e7-03ba-4a2e-bc90-720bc4689d0d None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:51:33.870869 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501159e7-03ba-4a2e-bc90-720bc4689d0d None None] PATH_INFO: `/` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:51:33.872210 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 668/1335] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 30 15:51:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 15:51:33.878343 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b0a251-97b0-4cd3-af6b-d5d426884aa8 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:51:33.878802 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b0a251-97b0-4cd3-af6b-d5d426884aa8 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:51:33.878908 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b0a251-97b0-4cd3-af6b-d5d426884aa8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:51:33.952742 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-76b0a251-97b0-4cd3-af6b-d5d426884aa8 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:51:33.953919 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-76b0a251-97b0-4cd3-af6b-d5d426884aa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:51:34.060273 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 668/1336] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 30 15:51:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:51:34.350408 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501159e7-03ba-4a2e-bc90-720bc4689d0d None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:51:34.352836 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-501159e7-03ba-4a2e-bc90-720bc4689d0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:51:34.403112 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:51:34.468774 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501159e7-03ba-4a2e-bc90-720bc4689d0d None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:51:34.549187 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0251b9-77a6-4bd4-9cf5-ba52bdbc3cb6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f4157105f3488f807b7475008ab722', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:51:34.549836 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0251b9-77a6-4bd4-9cf5-ba52bdbc3cb6 service neutron] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:51:34.550115 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0251b9-77a6-4bd4-9cf5-ba52bdbc3cb6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:51:34.550384 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0251b9-77a6-4bd4-9cf5-ba52bdbc3cb6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:51:34.552022 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-da0251b9-77a6-4bd4-9cf5-ba52bdbc3cb6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:51:34.659627 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:51:34.686522 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0251b9-77a6-4bd4-9cf5-ba52bdbc3cb6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:51:34.687978 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0251b9-77a6-4bd4-9cf5-ba52bdbc3cb6 service neutron] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:51:34.708687 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 669/1337] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 15:51:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:51:36.961910 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0bd225-da07-48bb-a016-a21403b94691 None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:51:36.962796 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0bd225-da07-48bb-a016-a21403b94691 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:51:36.962796 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0bd225-da07-48bb-a016-a21403b94691 None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:51:36.965067 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 669/1338] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 30 15:51:36 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 15:51:36.973116 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576b1110-36bf-40db-8e60-ea74fc9d70ab None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:51:36.973608 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576b1110-36bf-40db-8e60-ea74fc9d70ab None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:51:36.973702 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576b1110-36bf-40db-8e60-ea74fc9d70ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:51:37.041788 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-576b1110-36bf-40db-8e60-ea74fc9d70ab None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:51:37.043049 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-576b1110-36bf-40db-8e60-ea74fc9d70ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:51:37.142984 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 670/1339] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 30 15:51:36 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 15:51:37.425283 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0bd225-da07-48bb-a016-a21403b94691 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:51:37.427713 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-1f0bd225-da07-48bb-a016-a21403b94691 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:51:37.507399 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:51:37.539590 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0bd225-da07-48bb-a016-a21403b94691 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:51:37.624062 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3782cd66-e0d5-4131-8cbc-c2870e8f1050 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f4157105f3488f807b7475008ab722', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:51:37.624677 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3782cd66-e0d5-4131-8cbc-c2870e8f1050 service neutron] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:51:37.624945 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3782cd66-e0d5-4131-8cbc-c2870e8f1050 service neutron] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:51:37.625211 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3782cd66-e0d5-4131-8cbc-c2870e8f1050 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:51:37.626856 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-3782cd66-e0d5-4131-8cbc-c2870e8f1050 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:51:37.740936 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3782cd66-e0d5-4131-8cbc-c2870e8f1050 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:51:37.742349 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3782cd66-e0d5-4131-8cbc-c2870e8f1050 service neutron] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:51:37.760932 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 670/1340] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 15:51:37 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:51:41.651754 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d7b615-7dda-4178-84c2-511eb8393a92 None None] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:51:41.652222 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d7b615-7dda-4178-84c2-511eb8393a92 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:51:41.652380 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d7b615-7dda-4178-84c2-511eb8393a92 None None] PATH_INFO: `/` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:51:41.653801 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 671/1341] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 30 15:51:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 15:51:41.659240 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d102cc9-3d1f-481e-91d9-7388facfb5fb None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:51:41.659698 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d102cc9-3d1f-481e-91d9-7388facfb5fb None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:51:41.659807 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d102cc9-3d1f-481e-91d9-7388facfb5fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:51:41.665306 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:51:41.728360 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-0d102cc9-3d1f-481e-91d9-7388facfb5fb None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:51:41.729663 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-0d102cc9-3d1f-481e-91d9-7388facfb5fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:51:41.829850 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 671/1342] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 30 15:51:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:51:42.107397 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d7b615-7dda-4178-84c2-511eb8393a92 None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:51:42.108937 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-d4d7b615-7dda-4178-84c2-511eb8393a92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:51:42.159348 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:51:42.215825 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d7b615-7dda-4178-84c2-511eb8393a92 None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:51:42.295590 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88931549-523d-4b31-84a8-208b0969d610 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f4157105f3488f807b7475008ab722', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:51:42.296203 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88931549-523d-4b31-84a8-208b0969d610 service neutron] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:51:42.296487 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88931549-523d-4b31-84a8-208b0969d610 service neutron] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:51:42.296724 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88931549-523d-4b31-84a8-208b0969d610 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:51:42.298373 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-88931549-523d-4b31-84a8-208b0969d610 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:51:42.395219 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:51:42.412475 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88931549-523d-4b31-84a8-208b0969d610 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:51:42.413967 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88931549-523d-4b31-84a8-208b0969d610 service neutron] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:51:42.432254 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 672/1343] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 15:51:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:51:45.138134 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d35f0e5-30c0-4876-9f1b-ef338380b933 None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:51:45.138731 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d35f0e5-30c0-4876-9f1b-ef338380b933 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:51:45.138731 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d35f0e5-30c0-4876-9f1b-ef338380b933 None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:51:45.140186 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 672/1344] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 30 15:51:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 15:51:45.145551 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3f3882-fd53-41f4-bda4-6e104df81e69 None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:51:45.146021 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3f3882-fd53-41f4-bda4-6e104df81e69 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:51:45.146271 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3f3882-fd53-41f4-bda4-6e104df81e69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:51:45.216163 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-ad3f3882-fd53-41f4-bda4-6e104df81e69 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:51:45.217252 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-ad3f3882-fd53-41f4-bda4-6e104df81e69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:51:45.318183 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 673/1345] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 30 15:51:45 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:51:45.592463 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d35f0e5-30c0-4876-9f1b-ef338380b933 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:51:45.594324 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-5d35f0e5-30c0-4876-9f1b-ef338380b933 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:51:45.690460 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:51:45.708290 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d35f0e5-30c0-4876-9f1b-ef338380b933 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:51:45.786087 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f11ea42-3049-4fde-9ab8-0da556c22d11 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f4157105f3488f807b7475008ab722', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:51:45.786751 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f11ea42-3049-4fde-9ab8-0da556c22d11 service neutron] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:51:45.787026 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f11ea42-3049-4fde-9ab8-0da556c22d11 service neutron] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:51:45.787288 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f11ea42-3049-4fde-9ab8-0da556c22d11 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:51:45.788946 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-3f11ea42-3049-4fde-9ab8-0da556c22d11 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:51:45.905782 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f11ea42-3049-4fde-9ab8-0da556c22d11 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:51:45.907237 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f11ea42-3049-4fde-9ab8-0da556c22d11 service neutron] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:51:45.927379 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 673/1346] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 15:51:45 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:51:49.989172 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ca65aa-8083-45c8-971c-ffec405ee9df None None] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:51:49.989748 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ca65aa-8083-45c8-971c-ffec405ee9df None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:51:49.989748 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ca65aa-8083-45c8-971c-ffec405ee9df None None] PATH_INFO: `/` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:51:49.991328 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 674/1347] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 30 15:51:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 15:51:49.997011 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0457c201-0da4-4015-a6ac-86cf1449fcb3 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:51:49.997534 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0457c201-0da4-4015-a6ac-86cf1449fcb3 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:51:49.997661 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0457c201-0da4-4015-a6ac-86cf1449fcb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:51:50.013996 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:51:50.067871 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-0457c201-0da4-4015-a6ac-86cf1449fcb3 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:51:50.069181 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-0457c201-0da4-4015-a6ac-86cf1449fcb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:51:50.172769 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 674/1348] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 30 15:51:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:51:50.464991 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ca65aa-8083-45c8-971c-ffec405ee9df None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:51:50.467578 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-18ca65aa-8083-45c8-971c-ffec405ee9df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:51:50.531846 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:51:50.581459 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ca65aa-8083-45c8-971c-ffec405ee9df None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:51:50.661285 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c590495c-e621-4bb1-9a77-612a6ea45675 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f4157105f3488f807b7475008ab722', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:51:50.661911 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c590495c-e621-4bb1-9a77-612a6ea45675 service neutron] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:51:50.662188 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c590495c-e621-4bb1-9a77-612a6ea45675 service neutron] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:51:50.662457 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c590495c-e621-4bb1-9a77-612a6ea45675 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:51:50.664080 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-c590495c-e621-4bb1-9a77-612a6ea45675 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:51:50.783600 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:51:50.788270 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c590495c-e621-4bb1-9a77-612a6ea45675 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:51:50.789792 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c590495c-e621-4bb1-9a77-612a6ea45675 service neutron] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:51:50.808530 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 675/1349] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 15:51:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:51:55.738179 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70dc1e0-50f0-4873-bf1f-1da56e5f61fd None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:51:55.738179 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70dc1e0-50f0-4873-bf1f-1da56e5f61fd None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:51:55.738831 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70dc1e0-50f0-4873-bf1f-1da56e5f61fd None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:51:55.742911 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 675/1350] 10.222.0.20 () {54 vars in 921 bytes} [Tue Aug 30 15:51:55 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 15:51:55.749161 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ca1b57-73eb-4b60-b5e3-323131da6f04 None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:51:55.749806 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ca1b57-73eb-4b60-b5e3-323131da6f04 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:51:55.750164 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ca1b57-73eb-4b60-b5e3-323131da6f04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:51:55.858219 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-c7ca1b57-73eb-4b60-b5e3-323131da6f04 None None] MFA Rules not processed for user `bc12c6d31eab4a17a160250adc2380c8`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:51:55.865743 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-c7ca1b57-73eb-4b60-b5e3-323131da6f04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:51:56.025248 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 676/1351] 10.222.0.20 () {58 vars in 1018 bytes} [Tue Aug 30 15:51:55 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 276 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:52:00.171074 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=59 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:52:00.295728 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2dc80d-db66-4d29-9132-c65cf78d0336 None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:52:00.296150 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2dc80d-db66-4d29-9132-c65cf78d0336 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:52:00.296274 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2dc80d-db66-4d29-9132-c65cf78d0336 None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:52:00.297873 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 676/1352] 10.222.0.20 () {54 vars in 921 bytes} [Tue Aug 30 15:52:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 15:52:00.302639 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7137210-3d8b-46a2-9345-c14121286e38 None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:52:00.303140 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7137210-3d8b-46a2-9345-c14121286e38 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:52:00.303224 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7137210-3d8b-46a2-9345-c14121286e38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:52:00.359333 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:52:00.385957 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-a7137210-3d8b-46a2-9345-c14121286e38 None None] MFA Rules not processed for user `bc12c6d31eab4a17a160250adc2380c8`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:52:00.387519 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-a7137210-3d8b-46a2-9345-c14121286e38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:52:00.514669 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 677/1353] 10.222.0.20 () {58 vars in 1018 bytes} [Tue Aug 30 15:52:00 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 15:52:01.665468 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7429494c-a38e-4e0c-ace9-d1e78f21dfd8 None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:52:01.665976 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7429494c-a38e-4e0c-ace9-d1e78f21dfd8 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:52:01.665976 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7429494c-a38e-4e0c-ace9-d1e78f21dfd8 None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:52:01.667345 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 677/1354] 10.222.0.20 () {54 vars in 921 bytes} [Tue Aug 30 15:52:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 15:52:01.672849 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61961276-1a7b-4197-b425-5e974c6bd7d2 None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:52:01.673319 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61961276-1a7b-4197-b425-5e974c6bd7d2 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:52:01.673447 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61961276-1a7b-4197-b425-5e974c6bd7d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:52:01.751703 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-61961276-1a7b-4197-b425-5e974c6bd7d2 None None] MFA Rules not processed for user `bc12c6d31eab4a17a160250adc2380c8`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:52:01.752929 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-61961276-1a7b-4197-b425-5e974c6bd7d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:52:01.797770 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:52:01.863888 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 678/1355] 10.222.0.20 () {58 vars in 1018 bytes} [Tue Aug 30 15:52:01 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:52:05.182113 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c90b14-d53e-4c65-b279-87e71e7364eb None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:52:05.182721 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c90b14-d53e-4c65-b279-87e71e7364eb None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:52:05.182721 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c90b14-d53e-4c65-b279-87e71e7364eb None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:52:05.183977 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 678/1356] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 30 15:52:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 15:52:05.189252 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4fc989-5387-48d2-9392-3be71f2abcf8 None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:52:05.189725 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4fc989-5387-48d2-9392-3be71f2abcf8 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:52:05.189822 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4fc989-5387-48d2-9392-3be71f2abcf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:52:05.274667 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-ed4fc989-5387-48d2-9392-3be71f2abcf8 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:52:05.275873 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-ed4fc989-5387-48d2-9392-3be71f2abcf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:52:05.409688 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 679/1357] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 30 15:52:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:52:05.710722 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acd36c8-bcfe-401e-a9b0-a5544255168c None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:52:05.711294 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acd36c8-bcfe-401e-a9b0-a5544255168c None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:52:05.712150 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acd36c8-bcfe-401e-a9b0-a5544255168c None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:52:05.714019 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 679/1358] 10.222.0.20 () {54 vars in 962 bytes} [Tue Aug 30 15:52:05 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 15:52:05.722463 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0658790d-b235-4d90-a5b1-79cdb98932d1 None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:52:05.722772 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0658790d-b235-4d90-a5b1-79cdb98932d1 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:52:05.723040 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0658790d-b235-4d90-a5b1-79cdb98932d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:52:05.747030 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:52:05.814206 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-0658790d-b235-4d90-a5b1-79cdb98932d1 None None] MFA Rules not processed for user `7a1e756ae9be4a7bac1e81acf07e8f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:52:05.815649 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-0658790d-b235-4d90-a5b1-79cdb98932d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:52:05.947222 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 680/1359] 10.222.0.20 () {58 vars in 1059 bytes} [Tue Aug 30 15:52:05 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:52:05.955232 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acd36c8-bcfe-401e-a9b0-a5544255168c None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:52:05.956756 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-4acd36c8-bcfe-401e-a9b0-a5544255168c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:52:06.123514 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acd36c8-bcfe-401e-a9b0-a5544255168c None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:52:06.201689 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:52:06.246144 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b791c042-2ce5-4aac-9eb6-99f63c0ec683 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a1e756ae9be4a7bac1e81acf07e8f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:52:06.246729 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b791c042-2ce5-4aac-9eb6-99f63c0ec683 service nova] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:52:06.247003 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b791c042-2ce5-4aac-9eb6-99f63c0ec683 service nova] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:52:06.247299 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b791c042-2ce5-4aac-9eb6-99f63c0ec683 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:52:06.248894 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-b791c042-2ce5-4aac-9eb6-99f63c0ec683 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:52:06.343304 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b791c042-2ce5-4aac-9eb6-99f63c0ec683 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:52:06.344922 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b791c042-2ce5-4aac-9eb6-99f63c0ec683 service nova] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:52:06.363726 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 680/1360] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 15:52:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 15:52:08.482004 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad785e3-4c90-4269-95e9-402138321956 None None] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:52:08.482555 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad785e3-4c90-4269-95e9-402138321956 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:52:08.482555 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad785e3-4c90-4269-95e9-402138321956 None None] PATH_INFO: `/` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:52:08.484043 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 681/1361] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 30 15:52:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 15:52:08.489924 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d3aab2-9223-4b31-8f24-a202b11e3a11 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:52:08.490442 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d3aab2-9223-4b31-8f24-a202b11e3a11 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:52:08.490442 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d3aab2-9223-4b31-8f24-a202b11e3a11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:52:08.558679 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-d6d3aab2-9223-4b31-8f24-a202b11e3a11 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:52:08.559884 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-d6d3aab2-9223-4b31-8f24-a202b11e3a11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:52:08.599667 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:52:08.671829 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 681/1362] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 30 15:52:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:52:08.976379 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad785e3-4c90-4269-95e9-402138321956 None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:52:08.976379 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-1ad785e3-4c90-4269-95e9-402138321956 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:52:09.053463 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:52:09.137987 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad785e3-4c90-4269-95e9-402138321956 None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:52:09.254216 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c73e36-45a3-4605-8227-105e9cdd1bfc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a1e756ae9be4a7bac1e81acf07e8f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:52:09.254825 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c73e36-45a3-4605-8227-105e9cdd1bfc service nova] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:52:09.255094 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c73e36-45a3-4605-8227-105e9cdd1bfc service nova] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:52:09.255356 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c73e36-45a3-4605-8227-105e9cdd1bfc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:52:09.256941 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-76c73e36-45a3-4605-8227-105e9cdd1bfc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:52:09.283008 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:52:09.362671 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c73e36-45a3-4605-8227-105e9cdd1bfc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:52:09.364451 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c73e36-45a3-4605-8227-105e9cdd1bfc service nova] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:52:09.383789 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 682/1363] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 15:52:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:52:11.600780 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163ee64f-8fec-4e30-a3bf-f951344428b8 None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:52:11.601397 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163ee64f-8fec-4e30-a3bf-f951344428b8 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:52:11.601397 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163ee64f-8fec-4e30-a3bf-f951344428b8 None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:52:11.602919 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 682/1364] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 30 15:52:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 15:52:11.607989 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d633b4d7-0747-4355-8c35-2d80dfe82c4d None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:52:11.608405 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d633b4d7-0747-4355-8c35-2d80dfe82c4d None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:52:11.608583 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d633b4d7-0747-4355-8c35-2d80dfe82c4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:52:11.682220 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-d633b4d7-0747-4355-8c35-2d80dfe82c4d None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:52:11.683440 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-d633b4d7-0747-4355-8c35-2d80dfe82c4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:52:11.783774 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:52:11.786465 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 683/1365] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 30 15:52:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:52:12.074313 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c5047c-a9eb-4406-b203-ece93f2d5fe9 None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:52:12.074566 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c5047c-a9eb-4406-b203-ece93f2d5fe9 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:52:12.075040 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c5047c-a9eb-4406-b203-ece93f2d5fe9 None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:52:12.076335 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 683/1366] 10.222.0.20 () {54 vars in 962 bytes} [Tue Aug 30 15:52:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 15:52:12.082343 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147a8493-f7d2-447a-b337-4f519b871b62 None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:52:12.082633 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147a8493-f7d2-447a-b337-4f519b871b62 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:52:12.082870 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147a8493-f7d2-447a-b337-4f519b871b62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:52:12.168292 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-147a8493-f7d2-447a-b337-4f519b871b62 None None] MFA Rules not processed for user `7a1e756ae9be4a7bac1e81acf07e8f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:52:12.169532 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-147a8493-f7d2-447a-b337-4f519b871b62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:52:12.292480 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 684/1367] 10.222.0.20 () {58 vars in 1059 bytes} [Tue Aug 30 15:52:12 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:52:12.300466 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c5047c-a9eb-4406-b203-ece93f2d5fe9 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:52:12.302022 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-83c5047c-a9eb-4406-b203-ece93f2d5fe9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:52:12.456302 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c5047c-a9eb-4406-b203-ece93f2d5fe9 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:52:12.480877 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:52:12.580671 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b6ba58-b046-40ed-8df0-64ac5668a6da service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a1e756ae9be4a7bac1e81acf07e8f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:52:12.581324 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b6ba58-b046-40ed-8df0-64ac5668a6da service nova] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:52:12.581580 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b6ba58-b046-40ed-8df0-64ac5668a6da service nova] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:52:12.581871 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b6ba58-b046-40ed-8df0-64ac5668a6da service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:52:12.583452 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-43b6ba58-b046-40ed-8df0-64ac5668a6da service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:52:12.680814 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b6ba58-b046-40ed-8df0-64ac5668a6da service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:52:12.682629 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b6ba58-b046-40ed-8df0-64ac5668a6da service nova] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:52:12.702159 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 684/1368] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 15:52:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:52:13.318998 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfce7e91-fc61-4aa1-a7a4-e93977e49d6e None None] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:52:13.319613 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfce7e91-fc61-4aa1-a7a4-e93977e49d6e None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:52:13.319613 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfce7e91-fc61-4aa1-a7a4-e93977e49d6e None None] PATH_INFO: `/` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:52:13.321113 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 685/1369] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 30 15:52:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 15:52:13.326408 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bff3827-3f58-4112-986a-3f6b7fa4a2e2 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:52:13.326670 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bff3827-3f58-4112-986a-3f6b7fa4a2e2 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:52:13.326962 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bff3827-3f58-4112-986a-3f6b7fa4a2e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:52:13.360713 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:52:13.399443 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-6bff3827-3f58-4112-986a-3f6b7fa4a2e2 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:52:13.400901 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-6bff3827-3f58-4112-986a-3f6b7fa4a2e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:52:13.506633 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 685/1370] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 30 15:52:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:52:14.090218 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c13da2d-6146-4d65-9cbb-1fb0ab36d2f5 None None] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:52:14.090619 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c13da2d-6146-4d65-9cbb-1fb0ab36d2f5 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:52:14.090983 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c13da2d-6146-4d65-9cbb-1fb0ab36d2f5 None None] PATH_INFO: `/` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:52:14.092450 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 686/1371] 10.222.0.20 () {54 vars in 964 bytes} [Tue Aug 30 15:52:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 15:52:14.097630 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a12b83d-d986-47d0-896e-487807f58710 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:52:14.098118 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a12b83d-d986-47d0-896e-487807f58710 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:52:14.098199 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a12b83d-d986-47d0-896e-487807f58710 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:52:14.182495 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-3a12b83d-d986-47d0-896e-487807f58710 None None] MFA Rules not processed for user `0100a04821804993866ea20f159663f9`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:52:14.183733 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-3a12b83d-d986-47d0-896e-487807f58710 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:52:14.233346 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:52:14.266854 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 686/1372] 10.222.0.20 () {58 vars in 1061 bytes} [Tue Aug 30 15:52:14 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:52:14.274907 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c13da2d-6146-4d65-9cbb-1fb0ab36d2f5 None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:52:14.276544 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-8c13da2d-6146-4d65-9cbb-1fb0ab36d2f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:52:14.336053 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:52:14.411528 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c13da2d-6146-4d65-9cbb-1fb0ab36d2f5 None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:52:14.508354 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acd2679-c970-4b9b-be4f-adeb1165a97c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0100a04821804993866ea20f159663f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:52:14.509525 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acd2679-c970-4b9b-be4f-adeb1165a97c service cinder] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:52:14.509835 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acd2679-c970-4b9b-be4f-adeb1165a97c service cinder] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:52:14.510185 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acd2679-c970-4b9b-be4f-adeb1165a97c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:52:14.512002 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-8acd2679-c970-4b9b-be4f-adeb1165a97c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:52:14.651647 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8acd2679-c970-4b9b-be4f-adeb1165a97c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:52:14.654334 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8acd2679-c970-4b9b-be4f-adeb1165a97c service cinder] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:52:14.659669 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:52:14.681351 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 687/1373] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 15:52:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:52:14.716364 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9dfd44-ed9b-45b0-8280-32a7b59e542f None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:52:14.716700 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9dfd44-ed9b-45b0-8280-32a7b59e542f None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:52:14.717179 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9dfd44-ed9b-45b0-8280-32a7b59e542f None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:52:14.721731 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 687/1374] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 30 15:52:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 15:52:14.723824 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d023d04e-3858-4bea-b6c1-59973320dcb8 None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:52:14.724268 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d023d04e-3858-4bea-b6c1-59973320dcb8 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:52:14.726020 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d023d04e-3858-4bea-b6c1-59973320dcb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:52:14.800178 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-d023d04e-3858-4bea-b6c1-59973320dcb8 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:52:14.801776 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-d023d04e-3858-4bea-b6c1-59973320dcb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:52:14.920089 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 688/1375] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 30 15:52:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:52:15.241832 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9dfd44-ed9b-45b0-8280-32a7b59e542f None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:52:15.243550 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-7f9dfd44-ed9b-45b0-8280-32a7b59e542f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:52:15.399336 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9dfd44-ed9b-45b0-8280-32a7b59e542f None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:52:15.455314 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:52:15.518788 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7654ed20-95c8-4c81-b93b-bc21a2dae6dd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a1e756ae9be4a7bac1e81acf07e8f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:52:15.519325 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7654ed20-95c8-4c81-b93b-bc21a2dae6dd service nova] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:52:15.519600 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7654ed20-95c8-4c81-b93b-bc21a2dae6dd service nova] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:52:15.519865 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7654ed20-95c8-4c81-b93b-bc21a2dae6dd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:52:15.521513 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-7654ed20-95c8-4c81-b93b-bc21a2dae6dd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:52:15.613095 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7654ed20-95c8-4c81-b93b-bc21a2dae6dd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:52:15.614898 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7654ed20-95c8-4c81-b93b-bc21a2dae6dd service nova] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:52:15.634748 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 688/1376] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 15:52:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:52:17.583290 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9841437b-f43a-4ecb-abc6-3581e2b171b6 None None] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:52:17.583821 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9841437b-f43a-4ecb-abc6-3581e2b171b6 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:52:17.583821 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9841437b-f43a-4ecb-abc6-3581e2b171b6 None None] PATH_INFO: `/` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:52:17.585240 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 689/1377] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 30 15:52:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 15:52:17.590566 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91194a8f-54ba-41b3-b7aa-64581ee9b6f2 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:52:17.591021 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91194a8f-54ba-41b3-b7aa-64581ee9b6f2 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:52:17.591117 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91194a8f-54ba-41b3-b7aa-64581ee9b6f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:52:17.669461 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-91194a8f-54ba-41b3-b7aa-64581ee9b6f2 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:52:17.671613 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-91194a8f-54ba-41b3-b7aa-64581ee9b6f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:52:17.678942 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:52:17.791695 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 689/1378] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 30 15:52:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:52:18.079266 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9841437b-f43a-4ecb-abc6-3581e2b171b6 None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:52:18.080882 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-9841437b-f43a-4ecb-abc6-3581e2b171b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:52:18.156732 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:52:18.274663 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9841437b-f43a-4ecb-abc6-3581e2b171b6 None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:52:18.404356 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec94ddc-fb0c-457c-aee5-23b79d13624e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a1e756ae9be4a7bac1e81acf07e8f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:52:18.404963 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec94ddc-fb0c-457c-aee5-23b79d13624e service nova] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:52:18.405255 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec94ddc-fb0c-457c-aee5-23b79d13624e service nova] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:52:18.405541 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec94ddc-fb0c-457c-aee5-23b79d13624e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:52:18.407129 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-aec94ddc-fb0c-457c-aee5-23b79d13624e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:52:18.428569 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:52:18.512412 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec94ddc-fb0c-457c-aee5-23b79d13624e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:52:18.514298 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec94ddc-fb0c-457c-aee5-23b79d13624e service nova] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:52:18.535738 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 690/1379] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 15:52:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 457 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:52:20.467306 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fc19c0-4432-4bf1-8c25-99ff31c858f6 None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:52:20.467826 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fc19c0-4432-4bf1-8c25-99ff31c858f6 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:52:20.467826 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fc19c0-4432-4bf1-8c25-99ff31c858f6 None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:52:20.469446 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 690/1380] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 30 15:52:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 15:52:20.474444 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36bd0df-41a6-4973-aa8f-c0f409b0c4b8 None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:52:20.474867 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36bd0df-41a6-4973-aa8f-c0f409b0c4b8 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:52:20.475023 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36bd0df-41a6-4973-aa8f-c0f409b0c4b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:52:20.547202 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-f36bd0df-41a6-4973-aa8f-c0f409b0c4b8 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:52:20.548427 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-f36bd0df-41a6-4973-aa8f-c0f409b0c4b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:52:20.659580 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:52:20.667268 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 691/1381] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 30 15:52:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:52:20.953526 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fc19c0-4432-4bf1-8c25-99ff31c858f6 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:52:20.955075 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-48fc19c0-4432-4bf1-8c25-99ff31c858f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:52:21.116877 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:52:21.128739 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fc19c0-4432-4bf1-8c25-99ff31c858f6 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:52:21.253469 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d374aea6-8e17-46f4-b51c-217fc83da022 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a1e756ae9be4a7bac1e81acf07e8f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:52:21.253986 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d374aea6-8e17-46f4-b51c-217fc83da022 service nova] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:52:21.254260 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d374aea6-8e17-46f4-b51c-217fc83da022 service nova] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:52:21.254526 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d374aea6-8e17-46f4-b51c-217fc83da022 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:52:21.256139 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-d374aea6-8e17-46f4-b51c-217fc83da022 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:52:21.350172 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d374aea6-8e17-46f4-b51c-217fc83da022 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:52:21.351840 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d374aea6-8e17-46f4-b51c-217fc83da022 service nova] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:52:21.363349 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:52:21.371817 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 691/1382] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 15:52:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:52:23.651376 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d48245-a3fd-4d3d-a469-bf5f6c3d3fe4 None None] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:52:23.651937 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d48245-a3fd-4d3d-a469-bf5f6c3d3fe4 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:52:23.652054 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d48245-a3fd-4d3d-a469-bf5f6c3d3fe4 None None] PATH_INFO: `/` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:52:23.653634 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 692/1383] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 30 15:52:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 15:52:23.660008 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb90335-3145-4097-a83f-01cfe05ce66e None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:52:23.660465 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb90335-3145-4097-a83f-01cfe05ce66e None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:52:23.660679 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb90335-3145-4097-a83f-01cfe05ce66e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:52:23.739530 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-bdb90335-3145-4097-a83f-01cfe05ce66e None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:52:23.741104 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-bdb90335-3145-4097-a83f-01cfe05ce66e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:52:23.852523 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 692/1384] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 30 15:52:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 15:52:24.173769 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d48245-a3fd-4d3d-a469-bf5f6c3d3fe4 None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:52:24.175369 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-74d48245-a3fd-4d3d-a469-bf5f6c3d3fe4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:52:24.347996 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d48245-a3fd-4d3d-a469-bf5f6c3d3fe4 None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:52:24.416509 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:52:24.463832 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bac8dc-84fa-44ab-a776-a37993494110 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a1e756ae9be4a7bac1e81acf07e8f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:52:24.464451 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bac8dc-84fa-44ab-a776-a37993494110 service nova] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:52:24.464819 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bac8dc-84fa-44ab-a776-a37993494110 service nova] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:52:24.465149 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bac8dc-84fa-44ab-a776-a37993494110 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:52:24.466625 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-14bac8dc-84fa-44ab-a776-a37993494110 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:52:24.573826 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14bac8dc-84fa-44ab-a776-a37993494110 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:52:24.575679 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14bac8dc-84fa-44ab-a776-a37993494110 service nova] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:52:24.598136 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 693/1385] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 15:52:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:52:26.666653 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e95285-dfcc-494d-911d-d6d04f2d4f7f None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:52:26.667246 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e95285-dfcc-494d-911d-d6d04f2d4f7f None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:52:26.667246 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e95285-dfcc-494d-911d-d6d04f2d4f7f None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:52:26.668581 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 693/1386] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 30 15:52:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 15:52:26.675544 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b95b5d-19f9-4ee7-8885-1bd8f8e55238 None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:52:26.676051 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b95b5d-19f9-4ee7-8885-1bd8f8e55238 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:52:26.676291 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b95b5d-19f9-4ee7-8885-1bd8f8e55238 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:52:26.755849 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-62b95b5d-19f9-4ee7-8885-1bd8f8e55238 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:52:26.757120 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-62b95b5d-19f9-4ee7-8885-1bd8f8e55238 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:52:26.796216 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:52:26.868484 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 694/1387] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 30 15:52:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:52:27.156735 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e95285-dfcc-494d-911d-d6d04f2d4f7f None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:52:27.158318 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-56e95285-dfcc-494d-911d-d6d04f2d4f7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:52:27.239217 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:52:27.308369 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e95285-dfcc-494d-911d-d6d04f2d4f7f None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:52:27.424204 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32dafd9-5f75-49f4-8e01-9917025d9d94 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a1e756ae9be4a7bac1e81acf07e8f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:52:27.424732 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32dafd9-5f75-49f4-8e01-9917025d9d94 service nova] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:52:27.424992 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32dafd9-5f75-49f4-8e01-9917025d9d94 service nova] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:52:27.425290 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32dafd9-5f75-49f4-8e01-9917025d9d94 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:52:27.426901 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-c32dafd9-5f75-49f4-8e01-9917025d9d94 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:52:27.465023 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:52:27.522757 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32dafd9-5f75-49f4-8e01-9917025d9d94 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:52:27.525057 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32dafd9-5f75-49f4-8e01-9917025d9d94 service nova] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:52:27.546643 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 694/1388] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 15:52:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:52:29.694586 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783ce582-6ea2-4997-9164-f1d52aaab4c8 None None] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:52:29.695167 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783ce582-6ea2-4997-9164-f1d52aaab4c8 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:52:29.695167 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783ce582-6ea2-4997-9164-f1d52aaab4c8 None None] PATH_INFO: `/` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:52:29.696863 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 695/1389] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 30 15:52:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 15:52:29.702440 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2b1e93-9505-43e6-8937-67cc17247d68 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:52:29.703214 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2b1e93-9505-43e6-8937-67cc17247d68 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:52:29.703214 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2b1e93-9505-43e6-8937-67cc17247d68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:52:29.774522 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-5e2b1e93-9505-43e6-8937-67cc17247d68 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:52:29.775983 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-5e2b1e93-9505-43e6-8937-67cc17247d68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:52:29.884570 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 695/1390] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 30 15:52:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:52:30.173013 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783ce582-6ea2-4997-9164-f1d52aaab4c8 None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:52:30.174572 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-783ce582-6ea2-4997-9164-f1d52aaab4c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:52:30.336622 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783ce582-6ea2-4997-9164-f1d52aaab4c8 None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:52:30.355562 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:52:30.456780 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272cb68b-4e44-4647-abde-873cfe34c19d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a1e756ae9be4a7bac1e81acf07e8f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:52:30.457339 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272cb68b-4e44-4647-abde-873cfe34c19d service nova] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:52:30.457612 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272cb68b-4e44-4647-abde-873cfe34c19d service nova] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:52:30.457881 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272cb68b-4e44-4647-abde-873cfe34c19d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:52:30.459487 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-272cb68b-4e44-4647-abde-873cfe34c19d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:52:30.553136 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272cb68b-4e44-4647-abde-873cfe34c19d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:52:30.555695 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272cb68b-4e44-4647-abde-873cfe34c19d service nova] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:52:30.580241 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 696/1391] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 15:52:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:52:32.558355 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f49f74a-0d6a-4655-9336-8bee77ceccfc None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:52:32.558927 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f49f74a-0d6a-4655-9336-8bee77ceccfc None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:52:32.558927 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f49f74a-0d6a-4655-9336-8bee77ceccfc None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:52:32.560260 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 696/1392] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 30 15:52:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 15:52:32.565721 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6c6383-587b-46f1-8bc5-49a566ed65f1 None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:52:32.566206 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6c6383-587b-46f1-8bc5-49a566ed65f1 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:52:32.566316 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6c6383-587b-46f1-8bc5-49a566ed65f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:52:32.594244 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:52:32.639039 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-ba6c6383-587b-46f1-8bc5-49a566ed65f1 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:52:32.640261 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-ba6c6383-587b-46f1-8bc5-49a566ed65f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:52:32.739602 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 697/1393] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 30 15:52:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:52:33.022225 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f49f74a-0d6a-4655-9336-8bee77ceccfc None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:52:33.024195 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-5f49f74a-0d6a-4655-9336-8bee77ceccfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:52:33.031195 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5efc488-4529-40ad-ad19-aa7e2e428b03 None None] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:52:33.031478 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5efc488-4529-40ad-ad19-aa7e2e428b03 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:52:33.031745 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5efc488-4529-40ad-ad19-aa7e2e428b03 None None] PATH_INFO: `/` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:52:33.033106 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 698/1394] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 30 15:52:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 15:52:33.038168 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d301a622-c307-479f-8eba-a58e6ddba8f8 None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:52:33.038652 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d301a622-c307-479f-8eba-a58e6ddba8f8 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:52:33.038929 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d301a622-c307-479f-8eba-a58e6ddba8f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:52:33.039511 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:52:33.104061 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-d301a622-c307-479f-8eba-a58e6ddba8f8 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:52:33.105340 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-d301a622-c307-479f-8eba-a58e6ddba8f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:52:33.158030 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:52:33.183224 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f49f74a-0d6a-4655-9336-8bee77ceccfc None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:52:33.205310 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 699/1395] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 30 15:52:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 15:52:33.270933 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:52:33.297255 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199cd5c3-d1c4-4a17-9a78-05aa9a9ab186 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a1e756ae9be4a7bac1e81acf07e8f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:52:33.297892 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199cd5c3-d1c4-4a17-9a78-05aa9a9ab186 service nova] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:52:33.298167 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199cd5c3-d1c4-4a17-9a78-05aa9a9ab186 service nova] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:52:33.298449 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199cd5c3-d1c4-4a17-9a78-05aa9a9ab186 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:52:33.299990 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-199cd5c3-d1c4-4a17-9a78-05aa9a9ab186 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:52:33.394163 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199cd5c3-d1c4-4a17-9a78-05aa9a9ab186 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:52:33.395789 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199cd5c3-d1c4-4a17-9a78-05aa9a9ab186 service nova] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:52:33.414446 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 697/1396] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 15:52:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:52:33.508783 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49333420-edf8-40c2-88b1-7c5b401fddb9 None None] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:52:33.509070 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49333420-edf8-40c2-88b1-7c5b401fddb9 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:52:33.509378 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49333420-edf8-40c2-88b1-7c5b401fddb9 None None] PATH_INFO: `/` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:52:33.510866 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 700/1397] 10.222.0.20 () {54 vars in 967 bytes} [Tue Aug 30 15:52:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 15:52:33.516596 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa0de8f-9967-48fd-9c62-6b88d2bd9505 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:52:33.516849 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa0de8f-9967-48fd-9c62-6b88d2bd9505 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:52:33.517158 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa0de8f-9967-48fd-9c62-6b88d2bd9505 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:52:33.602068 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-2aa0de8f-9967-48fd-9c62-6b88d2bd9505 None None] MFA Rules not processed for user `b70e06a7d06d4199ab4fc36342ee1d68`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:52:33.603192 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-2aa0de8f-9967-48fd-9c62-6b88d2bd9505 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:52:33.646359 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:52:33.687737 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 698/1398] 10.222.0.20 () {58 vars in 1064 bytes} [Tue Aug 30 15:52:33 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:52:33.695523 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49333420-edf8-40c2-88b1-7c5b401fddb9 None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:52:33.697044 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-49333420-edf8-40c2-88b1-7c5b401fddb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:52:33.806068 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49333420-edf8-40c2-88b1-7c5b401fddb9 None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:52:33.879791 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01c21d2-8f30-4efd-911a-44497e87535a service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b70e06a7d06d4199ab4fc36342ee1d68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:52:33.880402 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01c21d2-8f30-4efd-911a-44497e87535a service glance] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:52:33.880677 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01c21d2-8f30-4efd-911a-44497e87535a service glance] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:52:33.880940 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01c21d2-8f30-4efd-911a-44497e87535a service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:52:33.882586 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-a01c21d2-8f30-4efd-911a-44497e87535a service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:52:33.886488 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:52:33.995125 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01c21d2-8f30-4efd-911a-44497e87535a service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:52:33.996576 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01c21d2-8f30-4efd-911a-44497e87535a service glance] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:52:34.015002 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 701/1399] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 15:52:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:52:34.779299 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c398627-167f-43c4-8447-201fd68e63f0 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:52:34.779780 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c398627-167f-43c4-8447-201fd68e63f0 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:52:34.779877 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c398627-167f-43c4-8447-201fd68e63f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:52:34.847132 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-1c398627-167f-43c4-8447-201fd68e63f0 None None] MFA Rules not processed for user `07130d598131443486c2bc1c6673c45b`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:52:34.848339 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-1c398627-167f-43c4-8447-201fd68e63f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:52:34.936245 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 699/1400] 10.222.0.20 () {58 vars in 1009 bytes} [Tue Aug 30 15:52:34 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:52:34.982435 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37c6acd-f024-4e52-ac80-1e71fea7372c None None] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:52:34.982932 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37c6acd-f024-4e52-ac80-1e71fea7372c None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:52:34.983022 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37c6acd-f024-4e52-ac80-1e71fea7372c None None] PATH_INFO: `/` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:52:34.984733 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 702/1401] 10.222.0.20 () {54 vars in 943 bytes} [Tue Aug 30 15:52:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 15:52:34.991136 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260569f8-66de-48a2-adf0-6a70731bf973 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:52:34.991421 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260569f8-66de-48a2-adf0-6a70731bf973 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:52:34.991809 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260569f8-66de-48a2-adf0-6a70731bf973 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:52:35.068370 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:52:35.076154 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-260569f8-66de-48a2-adf0-6a70731bf973 None None] MFA Rules not processed for user `8fdafdc39a594bb4b0aeaf2fbc84c2fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:52:35.077434 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-260569f8-66de-48a2-adf0-6a70731bf973 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:52:35.190230 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 700/1402] 10.222.0.20 () {58 vars in 1040 bytes} [Tue Aug 30 15:52:34 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:52:35.199994 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37c6acd-f024-4e52-ac80-1e71fea7372c None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:52:35.201553 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-d37c6acd-f024-4e52-ac80-1e71fea7372c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:52:35.306886 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:52:35.339091 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37c6acd-f024-4e52-ac80-1e71fea7372c None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:52:35.347020 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb419b3-8d50-47a9-92bc-f847fdea68d4 None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:52:35.347320 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb419b3-8d50-47a9-92bc-f847fdea68d4 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:52:35.347588 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb419b3-8d50-47a9-92bc-f847fdea68d4 None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:52:35.348920 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 701/1403] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 30 15:52:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 15:52:35.353920 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2edd281-915f-41e2-bc50-cdade80ef982 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:52:35.354201 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2edd281-915f-41e2-bc50-cdade80ef982 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:52:35.354473 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2edd281-915f-41e2-bc50-cdade80ef982 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:52:35.422469 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-d2edd281-915f-41e2-bc50-cdade80ef982 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:52:35.423667 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-d2edd281-915f-41e2-bc50-cdade80ef982 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:52:35.441349 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550fbab2-a72e-446e-adff-aa0aa5f4621e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8fdafdc39a594bb4b0aeaf2fbc84c2fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:52:35.441965 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550fbab2-a72e-446e-adff-aa0aa5f4621e service swift] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:52:35.442241 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550fbab2-a72e-446e-adff-aa0aa5f4621e service swift] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:52:35.442510 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550fbab2-a72e-446e-adff-aa0aa5f4621e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:52:35.444121 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-550fbab2-a72e-446e-adff-aa0aa5f4621e service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:52:35.486107 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:52:35.527429 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 702/1404] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 30 15:52:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:52:35.532385 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550fbab2-a72e-446e-adff-aa0aa5f4621e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:52:35.534059 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550fbab2-a72e-446e-adff-aa0aa5f4621e service swift] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:52:35.542118 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 703/1405] 10.222.0.20 () {60 vars in 1384 bytes} [Tue Aug 30 15:52:35 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 30 15:52:35.812766 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2edd281-915f-41e2-bc50-cdade80ef982 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:52:35.814773 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-d2edd281-915f-41e2-bc50-cdade80ef982 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:52:35.940253 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2edd281-915f-41e2-bc50-cdade80ef982 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:52:36.035485 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:52:36.117914 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0a33e9-f25e-4a2e-8c13-31842d4fb232 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a1e756ae9be4a7bac1e81acf07e8f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:52:36.118535 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0a33e9-f25e-4a2e-8c13-31842d4fb232 service nova] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:52:36.118808 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0a33e9-f25e-4a2e-8c13-31842d4fb232 service nova] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:52:36.119072 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0a33e9-f25e-4a2e-8c13-31842d4fb232 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:52:36.120706 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-2c0a33e9-f25e-4a2e-8c13-31842d4fb232 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:52:36.215755 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0a33e9-f25e-4a2e-8c13-31842d4fb232 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:52:36.217393 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0a33e9-f25e-4a2e-8c13-31842d4fb232 service nova] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:52:36.238348 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 703/1406] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 15:52:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:52:38.214533 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510b61f6-c479-44e4-a2e1-1000afb45100 None None] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:52:38.215116 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510b61f6-c479-44e4-a2e1-1000afb45100 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:52:38.215116 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510b61f6-c479-44e4-a2e1-1000afb45100 None None] PATH_INFO: `/` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:52:38.219427 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 704/1407] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 30 15:52:38 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 15:52:38.222804 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef176708-79d9-4a85-9992-3acb4e7491d0 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:52:38.223931 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef176708-79d9-4a85-9992-3acb4e7491d0 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:52:38.224368 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef176708-79d9-4a85-9992-3acb4e7491d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:52:38.300617 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-ef176708-79d9-4a85-9992-3acb4e7491d0 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:52:38.302003 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-ef176708-79d9-4a85-9992-3acb4e7491d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:52:38.349318 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:52:38.417259 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 704/1408] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 30 15:52:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:52:38.699471 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510b61f6-c479-44e4-a2e1-1000afb45100 None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:52:38.701169 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-510b61f6-c479-44e4-a2e1-1000afb45100 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:52:38.709533 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:52:38.868459 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510b61f6-c479-44e4-a2e1-1000afb45100 None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:52:38.953966 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:52:38.991331 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029e93e6-01d6-4d06-ba4d-2a7db53954e9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a1e756ae9be4a7bac1e81acf07e8f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:52:38.991997 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029e93e6-01d6-4d06-ba4d-2a7db53954e9 service nova] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:52:38.992269 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029e93e6-01d6-4d06-ba4d-2a7db53954e9 service nova] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:52:38.992533 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029e93e6-01d6-4d06-ba4d-2a7db53954e9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:52:38.994171 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-029e93e6-01d6-4d06-ba4d-2a7db53954e9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:52:39.089454 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029e93e6-01d6-4d06-ba4d-2a7db53954e9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:52:39.091078 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029e93e6-01d6-4d06-ba4d-2a7db53954e9 service nova] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:52:39.112445 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 705/1409] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 15:52:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:52:48.411565 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=26 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:52:49.110082 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=59 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:52:57.563286 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c770d04-33dd-4b9d-b38d-b585f0a54ebe None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:52:57.563870 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c770d04-33dd-4b9d-b38d-b585f0a54ebe None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:52:57.563870 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c770d04-33dd-4b9d-b38d-b585f0a54ebe None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:52:57.565148 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 705/1410] 10.222.0.20 () {54 vars in 918 bytes} [Tue Aug 30 15:52:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 15:52:57.570491 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16009478-98e0-4c95-9a39-6b104c005959 None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:52:57.570956 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16009478-98e0-4c95-9a39-6b104c005959 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:52:57.571058 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16009478-98e0-4c95-9a39-6b104c005959 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:52:57.659326 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-16009478-98e0-4c95-9a39-6b104c005959 None None] MFA Rules not processed for user `bc12c6d31eab4a17a160250adc2380c8`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:52:57.660535 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-16009478-98e0-4c95-9a39-6b104c005959 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:52:57.775876 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 706/1411] 10.222.0.20 () {58 vars in 1015 bytes} [Tue Aug 30 15:52:57 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:52:57.790835 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4f509b-b1a7-48c7-83ea-b74099a58ddc None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:52:57.791114 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4f509b-b1a7-48c7-83ea-b74099a58ddc None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:52:57.791381 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4f509b-b1a7-48c7-83ea-b74099a58ddc None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:52:57.792709 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 706/1412] 10.222.0.20 () {54 vars in 918 bytes} [Tue Aug 30 15:52:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 15:52:57.798455 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c83c27-39f3-4d6e-ad7d-a48bea8abd3d None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:52:57.798867 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c83c27-39f3-4d6e-ad7d-a48bea8abd3d None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:52:57.799219 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c83c27-39f3-4d6e-ad7d-a48bea8abd3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:52:57.880725 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:52:57.884327 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-a5c83c27-39f3-4d6e-ad7d-a48bea8abd3d None None] MFA Rules not processed for user `7a1e756ae9be4a7bac1e81acf07e8f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:52:57.886641 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-a5c83c27-39f3-4d6e-ad7d-a48bea8abd3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:52:58.012389 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 707/1413] 10.222.0.20 () {58 vars in 1015 bytes} [Tue Aug 30 15:52:57 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:53:00.404843 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1727de2e-6db7-49fc-b0c2-68324067374d None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:53:00.405448 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1727de2e-6db7-49fc-b0c2-68324067374d None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:53:00.405448 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1727de2e-6db7-49fc-b0c2-68324067374d None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:53:00.406659 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 707/1414] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 30 15:53:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 15:53:00.413149 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cec66d-c2da-41ef-b4c4-c6ce6b830e19 None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:53:00.413761 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cec66d-c2da-41ef-b4c4-c6ce6b830e19 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:53:00.413908 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cec66d-c2da-41ef-b4c4-c6ce6b830e19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:53:00.482923 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-84cec66d-c2da-41ef-b4c4-c6ce6b830e19 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:53:00.484495 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-84cec66d-c2da-41ef-b4c4-c6ce6b830e19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:53:00.540542 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:53:00.588176 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 708/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 30 15:53:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:53:00.886779 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1727de2e-6db7-49fc-b0c2-68324067374d None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:53:00.889387 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-1727de2e-6db7-49fc-b0c2-68324067374d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:53:01.045744 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1727de2e-6db7-49fc-b0c2-68324067374d None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:53:01.130356 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:53:01.173975 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fa4b2b-9ba0-4cdf-b87a-63396fd820c2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a1e756ae9be4a7bac1e81acf07e8f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:53:01.174591 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fa4b2b-9ba0-4cdf-b87a-63396fd820c2 service nova] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:53:01.174869 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fa4b2b-9ba0-4cdf-b87a-63396fd820c2 service nova] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:53:01.175112 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fa4b2b-9ba0-4cdf-b87a-63396fd820c2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:53:01.176743 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-60fa4b2b-9ba0-4cdf-b87a-63396fd820c2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:53:01.271742 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60fa4b2b-9ba0-4cdf-b87a-63396fd820c2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:53:01.273458 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60fa4b2b-9ba0-4cdf-b87a-63396fd820c2 service nova] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:53:01.293151 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 708/1416] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 15:53:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:53:02.193683 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667bb8bb-9fa2-438d-8769-5ac7f27163f9 None None] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:53:02.194260 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667bb8bb-9fa2-438d-8769-5ac7f27163f9 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:53:02.194260 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667bb8bb-9fa2-438d-8769-5ac7f27163f9 None None] PATH_INFO: `/` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:53:02.195761 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 709/1417] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 30 15:53:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 15:53:02.202020 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc15376f-109c-4ab7-82f1-40aa869c1086 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:53:02.202782 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc15376f-109c-4ab7-82f1-40aa869c1086 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:53:02.202968 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc15376f-109c-4ab7-82f1-40aa869c1086 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:53:02.283007 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-fc15376f-109c-4ab7-82f1-40aa869c1086 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:53:02.284200 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-fc15376f-109c-4ab7-82f1-40aa869c1086 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:53:02.314721 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:53:02.388671 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 709/1418] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 30 15:53:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:53:02.540011 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667bb8bb-9fa2-438d-8769-5ac7f27163f9 None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:53:02.541478 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-667bb8bb-9fa2-438d-8769-5ac7f27163f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:53:02.696009 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667bb8bb-9fa2-438d-8769-5ac7f27163f9 None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:53:02.739944 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:53:02.822221 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108ea0c7-e77b-45be-8dd0-d429b769bd3e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a1e756ae9be4a7bac1e81acf07e8f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:53:02.822741 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108ea0c7-e77b-45be-8dd0-d429b769bd3e service nova] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:53:02.823011 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108ea0c7-e77b-45be-8dd0-d429b769bd3e service nova] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:53:02.823279 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108ea0c7-e77b-45be-8dd0-d429b769bd3e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:53:02.824882 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-108ea0c7-e77b-45be-8dd0-d429b769bd3e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:53:02.916734 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108ea0c7-e77b-45be-8dd0-d429b769bd3e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:53:02.918382 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108ea0c7-e77b-45be-8dd0-d429b769bd3e service nova] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:53:02.937787 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 710/1419] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 15:53:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:53:03.828550 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dcda0a-0002-4541-ab4c-5150fb43a0fa None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:53:03.829153 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dcda0a-0002-4541-ab4c-5150fb43a0fa None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:53:03.829153 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dcda0a-0002-4541-ab4c-5150fb43a0fa None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:53:03.830436 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 710/1420] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 30 15:53:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 15:53:03.836729 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8426955d-dc6c-48ad-a853-19630ff648ab None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:53:03.837513 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8426955d-dc6c-48ad-a853-19630ff648ab None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:53:03.837672 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8426955d-dc6c-48ad-a853-19630ff648ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:53:03.906049 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:53:03.933815 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-8426955d-dc6c-48ad-a853-19630ff648ab None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:53:03.935335 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-8426955d-dc6c-48ad-a853-19630ff648ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:53:04.039654 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 711/1421] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 30 15:53:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:53:04.194002 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9dcda0a-0002-4541-ab4c-5150fb43a0fa None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:53:04.196415 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-e9dcda0a-0002-4541-ab4c-5150fb43a0fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:53:04.354484 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9dcda0a-0002-4541-ab4c-5150fb43a0fa None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:53:04.367126 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:53:04.476867 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bfc1f0-df98-469e-86c7-7af6f1758a1b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a1e756ae9be4a7bac1e81acf07e8f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:53:04.477398 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bfc1f0-df98-469e-86c7-7af6f1758a1b service nova] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:53:04.477677 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bfc1f0-df98-469e-86c7-7af6f1758a1b service nova] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:53:04.477967 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bfc1f0-df98-469e-86c7-7af6f1758a1b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:53:04.479615 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-d8bfc1f0-df98-469e-86c7-7af6f1758a1b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:53:04.573810 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bfc1f0-df98-469e-86c7-7af6f1758a1b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:53:04.575428 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bfc1f0-df98-469e-86c7-7af6f1758a1b service nova] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:53:04.596363 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 711/1422] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 15:53:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:53:05.473909 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5749e2-485d-4dbb-94a0-8d83a91478e3 None None] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:53:05.474801 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5749e2-485d-4dbb-94a0-8d83a91478e3 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:53:05.474801 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5749e2-485d-4dbb-94a0-8d83a91478e3 None None] PATH_INFO: `/` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:53:05.481021 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 712/1423] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 30 15:53:05 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 15:53:05.489534 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb97a05f-1c62-49c1-958a-6bc6cc366f95 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:53:05.490343 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb97a05f-1c62-49c1-958a-6bc6cc366f95 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:53:05.490528 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb97a05f-1c62-49c1-958a-6bc6cc366f95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:53:05.516888 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:53:05.582011 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-eb97a05f-1c62-49c1-958a-6bc6cc366f95 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:53:05.583241 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-eb97a05f-1c62-49c1-958a-6bc6cc366f95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:53:05.688153 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 712/1424] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 30 15:53:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:53:05.844427 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5749e2-485d-4dbb-94a0-8d83a91478e3 None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:53:05.845890 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-6f5749e2-485d-4dbb-94a0-8d83a91478e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:53:05.966471 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:53:06.011635 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5749e2-485d-4dbb-94a0-8d83a91478e3 None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:53:06.133242 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76286f12-9f97-412a-ad9c-3549a8743331 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a1e756ae9be4a7bac1e81acf07e8f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:53:06.133857 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76286f12-9f97-412a-ad9c-3549a8743331 service nova] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:53:06.134136 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76286f12-9f97-412a-ad9c-3549a8743331 service nova] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:53:06.134406 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76286f12-9f97-412a-ad9c-3549a8743331 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:53:06.136389 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-76286f12-9f97-412a-ad9c-3549a8743331 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:53:06.217971 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:53:06.232556 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76286f12-9f97-412a-ad9c-3549a8743331 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:53:06.234555 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76286f12-9f97-412a-ad9c-3549a8743331 service nova] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:53:06.252837 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 713/1425] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 15:53:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:53:07.113296 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee771a6-e22f-43a8-ab01-9fb9589b711a None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:53:07.114297 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee771a6-e22f-43a8-ab01-9fb9589b711a None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:53:07.114297 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee771a6-e22f-43a8-ab01-9fb9589b711a None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:53:07.116393 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 713/1426] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 30 15:53:07 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 15:53:07.122693 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3027e2e0-325e-40c7-84c6-78c9e63f9e1d None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:53:07.123431 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3027e2e0-325e-40c7-84c6-78c9e63f9e1d None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:53:07.123588 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3027e2e0-325e-40c7-84c6-78c9e63f9e1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:53:07.206121 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-3027e2e0-325e-40c7-84c6-78c9e63f9e1d None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:53:07.207377 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-3027e2e0-325e-40c7-84c6-78c9e63f9e1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:53:07.308587 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 714/1427] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 30 15:53:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:53:07.460633 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee771a6-e22f-43a8-ab01-9fb9589b711a None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:53:07.462256 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-eee771a6-e22f-43a8-ab01-9fb9589b711a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:53:07.565866 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:53:07.641752 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee771a6-e22f-43a8-ab01-9fb9589b711a None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:53:07.763242 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101a28bd-9a89-4d73-8694-763ec0cc78c2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a1e756ae9be4a7bac1e81acf07e8f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:53:07.763773 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101a28bd-9a89-4d73-8694-763ec0cc78c2 service nova] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:53:07.764049 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101a28bd-9a89-4d73-8694-763ec0cc78c2 service nova] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:53:07.764313 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101a28bd-9a89-4d73-8694-763ec0cc78c2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:53:07.765962 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-101a28bd-9a89-4d73-8694-763ec0cc78c2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:53:07.815753 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:53:07.863934 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101a28bd-9a89-4d73-8694-763ec0cc78c2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:53:07.865578 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101a28bd-9a89-4d73-8694-763ec0cc78c2 service nova] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:53:07.883770 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 714/1428] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 15:53:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:53:08.696016 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab1bf15-2ddd-4b44-b1b9-7f35aeb698f9 None None] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:53:08.696500 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab1bf15-2ddd-4b44-b1b9-7f35aeb698f9 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:53:08.696682 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab1bf15-2ddd-4b44-b1b9-7f35aeb698f9 None None] PATH_INFO: `/` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:53:08.698957 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 715/1429] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 30 15:53:08 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 15:53:08.704170 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f0327a-d53a-41bf-a4fe-6ac1c4900f6a None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:53:08.704605 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f0327a-d53a-41bf-a4fe-6ac1c4900f6a None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:53:08.704699 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f0327a-d53a-41bf-a4fe-6ac1c4900f6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:53:08.785080 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-b4f0327a-d53a-41bf-a4fe-6ac1c4900f6a None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:53:08.786316 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-b4f0327a-d53a-41bf-a4fe-6ac1c4900f6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:53:08.890331 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 715/1430] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 30 15:53:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:53:09.045899 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab1bf15-2ddd-4b44-b1b9-7f35aeb698f9 None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:53:09.047463 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-5ab1bf15-2ddd-4b44-b1b9-7f35aeb698f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:53:09.110593 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:53:09.217709 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab1bf15-2ddd-4b44-b1b9-7f35aeb698f9 None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:53:09.345588 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ec92cb-7eaf-4d4a-964a-d60e35a42680 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a1e756ae9be4a7bac1e81acf07e8f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:53:09.346200 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ec92cb-7eaf-4d4a-964a-d60e35a42680 service nova] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:53:09.346473 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ec92cb-7eaf-4d4a-964a-d60e35a42680 service nova] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:53:09.346742 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ec92cb-7eaf-4d4a-964a-d60e35a42680 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:53:09.348339 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-f5ec92cb-7eaf-4d4a-964a-d60e35a42680 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:53:09.355726 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:53:09.442422 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ec92cb-7eaf-4d4a-964a-d60e35a42680 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:53:09.444066 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ec92cb-7eaf-4d4a-964a-d60e35a42680 service nova] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:53:09.462625 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 716/1431] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 15:53:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:53:10.266350 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdeaa394-a0e3-44a4-84f0-9d76c9d58ab8 None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:53:10.266931 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdeaa394-a0e3-44a4-84f0-9d76c9d58ab8 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:53:10.266931 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdeaa394-a0e3-44a4-84f0-9d76c9d58ab8 None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:53:10.268229 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 716/1432] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 30 15:53:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 15:53:10.274551 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8888c26-18b4-4f19-b06c-bc4035399cd5 None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:53:10.275308 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8888c26-18b4-4f19-b06c-bc4035399cd5 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:53:10.275308 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8888c26-18b4-4f19-b06c-bc4035399cd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:53:10.355249 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-c8888c26-18b4-4f19-b06c-bc4035399cd5 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:53:10.356440 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-c8888c26-18b4-4f19-b06c-bc4035399cd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:53:10.432382 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:53:10.464065 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 717/1433] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 30 15:53:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:53:10.617397 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdeaa394-a0e3-44a4-84f0-9d76c9d58ab8 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:53:10.618960 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-cdeaa394-a0e3-44a4-84f0-9d76c9d58ab8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:53:10.637765 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:53:10.768358 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdeaa394-a0e3-44a4-84f0-9d76c9d58ab8 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:53:10.865939 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:53:10.894255 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f6962b-7400-4438-9ae3-2c7dcb816425 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a1e756ae9be4a7bac1e81acf07e8f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:53:10.894873 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f6962b-7400-4438-9ae3-2c7dcb816425 service nova] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:53:10.895144 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f6962b-7400-4438-9ae3-2c7dcb816425 service nova] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:53:10.895379 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f6962b-7400-4438-9ae3-2c7dcb816425 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:53:10.896994 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-65f6962b-7400-4438-9ae3-2c7dcb816425 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:53:10.998046 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f6962b-7400-4438-9ae3-2c7dcb816425 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:53:10.999647 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f6962b-7400-4438-9ae3-2c7dcb816425 service nova] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:53:11.018002 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 717/1434] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 15:53:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:53:11.829253 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979d079c-9ec6-4bc9-8e7d-e45f5d032421 None None] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:53:11.829813 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979d079c-9ec6-4bc9-8e7d-e45f5d032421 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:53:11.829813 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979d079c-9ec6-4bc9-8e7d-e45f5d032421 None None] PATH_INFO: `/` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:53:11.831158 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 718/1435] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 30 15:53:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 15:53:11.837241 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c6fe87-c493-4c10-a272-55ef99653906 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:53:11.838664 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c6fe87-c493-4c10-a272-55ef99653906 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:53:11.839041 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c6fe87-c493-4c10-a272-55ef99653906 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:53:11.929402 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-e6c6fe87-c493-4c10-a272-55ef99653906 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:53:11.930553 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-e6c6fe87-c493-4c10-a272-55ef99653906 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:53:11.976185 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:53:12.032845 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 718/1436] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 30 15:53:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:53:12.187348 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979d079c-9ec6-4bc9-8e7d-e45f5d032421 None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:53:12.189741 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-979d079c-9ec6-4bc9-8e7d-e45f5d032421 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:53:12.347625 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979d079c-9ec6-4bc9-8e7d-e45f5d032421 None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:53:12.402990 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:53:12.467629 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580f34c3-5be6-46fe-ad31-7f6afdbb2867 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a1e756ae9be4a7bac1e81acf07e8f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:53:12.468160 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580f34c3-5be6-46fe-ad31-7f6afdbb2867 service nova] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:53:12.468441 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580f34c3-5be6-46fe-ad31-7f6afdbb2867 service nova] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:53:12.468717 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580f34c3-5be6-46fe-ad31-7f6afdbb2867 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:53:12.470360 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-580f34c3-5be6-46fe-ad31-7f6afdbb2867 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:53:12.589838 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580f34c3-5be6-46fe-ad31-7f6afdbb2867 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:53:12.591462 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580f34c3-5be6-46fe-ad31-7f6afdbb2867 service nova] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:53:12.611376 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 719/1437] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 15:53:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:53:13.411084 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d13665-aded-4115-b03a-2b1b8b73080c None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:53:13.412032 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d13665-aded-4115-b03a-2b1b8b73080c None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:53:13.412032 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d13665-aded-4115-b03a-2b1b8b73080c None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:53:13.414136 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 719/1438] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 30 15:53:13 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 15:53:13.419669 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544b7af9-2592-4d74-ab65-1c43d6f66b2d None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:53:13.420145 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544b7af9-2592-4d74-ab65-1c43d6f66b2d None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:53:13.420262 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544b7af9-2592-4d74-ab65-1c43d6f66b2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:53:13.498550 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-544b7af9-2592-4d74-ab65-1c43d6f66b2d None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:53:13.499756 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-544b7af9-2592-4d74-ab65-1c43d6f66b2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:53:13.501889 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:53:13.611468 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 720/1439] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 30 15:53:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:53:13.766247 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d13665-aded-4115-b03a-2b1b8b73080c None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:53:13.767683 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-e6d13665-aded-4115-b03a-2b1b8b73080c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:53:13.924623 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d13665-aded-4115-b03a-2b1b8b73080c None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:53:13.953483 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:53:14.058218 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83368ba-544c-4494-8a91-bb0ad6153984 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a1e756ae9be4a7bac1e81acf07e8f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:53:14.058738 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83368ba-544c-4494-8a91-bb0ad6153984 service nova] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:53:14.059018 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83368ba-544c-4494-8a91-bb0ad6153984 service nova] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:53:14.059268 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83368ba-544c-4494-8a91-bb0ad6153984 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:53:14.060870 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-e83368ba-544c-4494-8a91-bb0ad6153984 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:53:14.164155 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83368ba-544c-4494-8a91-bb0ad6153984 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:53:14.165855 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83368ba-544c-4494-8a91-bb0ad6153984 service nova] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:53:14.184087 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 720/1440] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 15:53:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:53:16.062504 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70e70ff-4a9c-4470-b6c5-0414f064d20a None None] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:53:16.063025 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70e70ff-4a9c-4470-b6c5-0414f064d20a None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:53:16.063025 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70e70ff-4a9c-4470-b6c5-0414f064d20a None None] PATH_INFO: `/` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:53:16.064397 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 721/1441] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 30 15:53:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 15:53:16.070051 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba887d64-42d0-4de8-bc84-ec03788f060c None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:53:16.070457 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba887d64-42d0-4de8-bc84-ec03788f060c None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:53:16.070634 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba887d64-42d0-4de8-bc84-ec03788f060c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:53:16.114006 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:53:16.143115 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-ba887d64-42d0-4de8-bc84-ec03788f060c None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:53:16.144441 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-ba887d64-42d0-4de8-bc84-ec03788f060c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:53:16.239964 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 721/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 30 15:53:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:53:16.527590 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70e70ff-4a9c-4470-b6c5-0414f064d20a None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:53:16.529268 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-e70e70ff-4a9c-4470-b6c5-0414f064d20a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:53:16.638563 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70e70ff-4a9c-4470-b6c5-0414f064d20a None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:53:16.671982 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:53:16.715016 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4422eff-8a9a-40b9-8356-0664dccacc79 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f4157105f3488f807b7475008ab722', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:53:16.715623 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4422eff-8a9a-40b9-8356-0664dccacc79 service neutron] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:53:16.715898 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4422eff-8a9a-40b9-8356-0664dccacc79 service neutron] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:53:16.716157 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4422eff-8a9a-40b9-8356-0664dccacc79 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:53:16.717794 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-d4422eff-8a9a-40b9-8356-0664dccacc79 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:53:16.832930 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4422eff-8a9a-40b9-8356-0664dccacc79 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:53:16.834338 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4422eff-8a9a-40b9-8356-0664dccacc79 service neutron] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:53:16.853587 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 722/1443] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 15:53:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:53:18.903207 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57261b36-1c58-46fe-ac02-cf0837e1fb9e None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:53:18.904171 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57261b36-1c58-46fe-ac02-cf0837e1fb9e None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:53:18.904171 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57261b36-1c58-46fe-ac02-cf0837e1fb9e None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:53:18.906264 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 722/1444] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 30 15:53:18 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 15:53:18.912680 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbee283-77e9-4da8-a7af-3eec0deee621 None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:53:18.913387 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbee283-77e9-4da8-a7af-3eec0deee621 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:53:18.913549 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbee283-77e9-4da8-a7af-3eec0deee621 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:53:18.982756 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-ffbee283-77e9-4da8-a7af-3eec0deee621 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:53:18.983954 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-ffbee283-77e9-4da8-a7af-3eec0deee621 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:53:18.997281 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:53:19.088366 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 723/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 30 15:53:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:53:19.374017 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57261b36-1c58-46fe-ac02-cf0837e1fb9e None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:53:19.375615 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-57261b36-1c58-46fe-ac02-cf0837e1fb9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:53:19.492964 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57261b36-1c58-46fe-ac02-cf0837e1fb9e None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:53:19.506683 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:53:19.577641 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9475b74e-9d49-489a-b97a-dc0badfccd5b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f4157105f3488f807b7475008ab722', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:53:19.578246 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9475b74e-9d49-489a-b97a-dc0badfccd5b service neutron] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:53:19.578522 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9475b74e-9d49-489a-b97a-dc0badfccd5b service neutron] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:53:19.578784 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9475b74e-9d49-489a-b97a-dc0badfccd5b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:53:19.580384 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-9475b74e-9d49-489a-b97a-dc0badfccd5b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:53:19.709985 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9475b74e-9d49-489a-b97a-dc0badfccd5b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:53:19.711508 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9475b74e-9d49-489a-b97a-dc0badfccd5b service neutron] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:53:19.731245 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 723/1446] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 15:53:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:53:22.021018 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e5ee50-0e14-4db8-bbd2-7d54ad3b7caf None None] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:53:22.021892 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e5ee50-0e14-4db8-bbd2-7d54ad3b7caf None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:53:22.021892 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e5ee50-0e14-4db8-bbd2-7d54ad3b7caf None None] PATH_INFO: `/` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:53:22.023852 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 724/1447] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 30 15:53:22 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 15:53:22.031415 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df93837-c501-4993-ac15-5b5e3788917b None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:53:22.031844 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df93837-c501-4993-ac15-5b5e3788917b None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:53:22.031953 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df93837-c501-4993-ac15-5b5e3788917b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:53:22.086507 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:53:22.107315 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-4df93837-c501-4993-ac15-5b5e3788917b None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:53:22.108538 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-4df93837-c501-4993-ac15-5b5e3788917b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:53:22.208111 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 724/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 30 15:53:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:53:22.491560 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e5ee50-0e14-4db8-bbd2-7d54ad3b7caf None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:53:22.494105 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-09e5ee50-0e14-4db8-bbd2-7d54ad3b7caf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:53:22.605998 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e5ee50-0e14-4db8-bbd2-7d54ad3b7caf None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:53:22.647673 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:53:22.685628 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3df69a1-dda3-4aee-b186-0dad8893a94c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f4157105f3488f807b7475008ab722', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:53:22.686222 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3df69a1-dda3-4aee-b186-0dad8893a94c service neutron] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:53:22.686494 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3df69a1-dda3-4aee-b186-0dad8893a94c service neutron] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:53:22.686753 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3df69a1-dda3-4aee-b186-0dad8893a94c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:53:22.688320 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-f3df69a1-dda3-4aee-b186-0dad8893a94c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:53:22.934672 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3df69a1-dda3-4aee-b186-0dad8893a94c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:53:22.936172 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3df69a1-dda3-4aee-b186-0dad8893a94c service neutron] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:53:22.955017 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 725/1449] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 15:53:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 465 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:53:25.001102 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0dce8d-0a87-4216-89e8-5f722e2bc088 None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:53:25.001829 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0dce8d-0a87-4216-89e8-5f722e2bc088 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:53:25.001829 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0dce8d-0a87-4216-89e8-5f722e2bc088 None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:53:25.003607 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 725/1450] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 30 15:53:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 15:53:25.009829 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7452a03-68ec-44a2-842d-df990660d4b1 None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:53:25.010282 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7452a03-68ec-44a2-842d-df990660d4b1 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:53:25.010399 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7452a03-68ec-44a2-842d-df990660d4b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:53:25.076166 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-d7452a03-68ec-44a2-842d-df990660d4b1 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:53:25.077422 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-d7452a03-68ec-44a2-842d-df990660d4b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:53:25.098587 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:53:25.181535 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 726/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 30 15:53:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:53:25.472155 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0dce8d-0a87-4216-89e8-5f722e2bc088 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:53:25.473752 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-5e0dce8d-0a87-4216-89e8-5f722e2bc088 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:53:25.580415 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0dce8d-0a87-4216-89e8-5f722e2bc088 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:53:25.601862 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:53:25.670477 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcde67b-4a41-47f3-99e5-f21008910d78 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f4157105f3488f807b7475008ab722', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:53:25.671534 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcde67b-4a41-47f3-99e5-f21008910d78 service neutron] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:53:25.672041 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcde67b-4a41-47f3-99e5-f21008910d78 service neutron] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:53:25.672539 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcde67b-4a41-47f3-99e5-f21008910d78 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:53:25.675735 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-ddcde67b-4a41-47f3-99e5-f21008910d78 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:53:25.802233 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddcde67b-4a41-47f3-99e5-f21008910d78 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:53:25.804311 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddcde67b-4a41-47f3-99e5-f21008910d78 service neutron] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:53:25.832332 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 726/1452] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 15:53:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:53:27.793284 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1396c9d0-a672-4bf7-b28c-079f62f919d5 None None] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:53:27.794154 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1396c9d0-a672-4bf7-b28c-079f62f919d5 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:53:27.794154 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1396c9d0-a672-4bf7-b28c-079f62f919d5 None None] PATH_INFO: `/` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:53:27.795952 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 727/1453] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 30 15:53:27 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 15:53:27.801451 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be319074-3aac-4545-91ba-69600dc37090 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:53:27.801936 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be319074-3aac-4545-91ba-69600dc37090 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:53:27.802046 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be319074-3aac-4545-91ba-69600dc37090 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:53:27.887868 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:53:27.891015 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-be319074-3aac-4545-91ba-69600dc37090 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:53:27.892831 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-be319074-3aac-4545-91ba-69600dc37090 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:53:28.001332 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 727/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 30 15:53:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:53:28.285441 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1396c9d0-a672-4bf7-b28c-079f62f919d5 None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:53:28.287903 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-1396c9d0-a672-4bf7-b28c-079f62f919d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:53:28.414024 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1396c9d0-a672-4bf7-b28c-079f62f919d5 None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:53:28.469992 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:53:28.496658 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5fcd2a-9da4-4a21-ac97-0cdeb8b67e1f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f4157105f3488f807b7475008ab722', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:53:28.497298 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5fcd2a-9da4-4a21-ac97-0cdeb8b67e1f service neutron] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:53:28.497570 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5fcd2a-9da4-4a21-ac97-0cdeb8b67e1f service neutron] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:53:28.497837 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5fcd2a-9da4-4a21-ac97-0cdeb8b67e1f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:53:28.499465 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-1d5fcd2a-9da4-4a21-ac97-0cdeb8b67e1f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:53:28.615798 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5fcd2a-9da4-4a21-ac97-0cdeb8b67e1f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:53:28.617269 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5fcd2a-9da4-4a21-ac97-0cdeb8b67e1f service neutron] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:53:28.635889 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 728/1455] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 15:53:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:53:30.593109 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f736405-8b1f-479f-8635-61527b42582d None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:53:30.593760 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f736405-8b1f-479f-8635-61527b42582d None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:53:30.593760 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f736405-8b1f-479f-8635-61527b42582d None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:53:30.595016 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 728/1456] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 30 15:53:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 15:53:30.601467 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10db935e-72cb-46a6-b43b-19826c17d666 None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:53:30.601915 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10db935e-72cb-46a6-b43b-19826c17d666 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:53:30.602022 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10db935e-72cb-46a6-b43b-19826c17d666 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:53:30.677818 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-10db935e-72cb-46a6-b43b-19826c17d666 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:53:30.679029 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-10db935e-72cb-46a6-b43b-19826c17d666 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:53:30.709566 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:53:30.779805 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 729/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 30 15:53:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:53:31.061530 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f736405-8b1f-479f-8635-61527b42582d None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:53:31.063088 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-8f736405-8b1f-479f-8635-61527b42582d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:53:31.173660 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f736405-8b1f-479f-8635-61527b42582d None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:53:31.203558 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:53:31.256998 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7a742b-3784-4eb7-9720-f87ece09b69a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f4157105f3488f807b7475008ab722', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:53:31.257645 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7a742b-3784-4eb7-9720-f87ece09b69a service neutron] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:53:31.257923 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7a742b-3784-4eb7-9720-f87ece09b69a service neutron] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:53:31.258196 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7a742b-3784-4eb7-9720-f87ece09b69a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:53:31.259804 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-5c7a742b-3784-4eb7-9720-f87ece09b69a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:53:31.373800 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7a742b-3784-4eb7-9720-f87ece09b69a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:53:31.375146 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7a742b-3784-4eb7-9720-f87ece09b69a service neutron] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:53:31.393461 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 729/1458] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 15:53:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:53:33.340717 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e115e48-4d3c-4f6e-a5e2-7f8d42ff95dc None None] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:53:33.341332 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e115e48-4d3c-4f6e-a5e2-7f8d42ff95dc None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:53:33.341332 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e115e48-4d3c-4f6e-a5e2-7f8d42ff95dc None None] PATH_INFO: `/` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:53:33.343382 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 730/1459] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 30 15:53:33 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 15:53:33.352173 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727c04ac-0d3c-44bf-bac8-8e36ca431801 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:53:33.352593 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727c04ac-0d3c-44bf-bac8-8e36ca431801 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:53:33.352726 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727c04ac-0d3c-44bf-bac8-8e36ca431801 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:53:33.431222 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-727c04ac-0d3c-44bf-bac8-8e36ca431801 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:53:33.432455 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-727c04ac-0d3c-44bf-bac8-8e36ca431801 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:53:33.441644 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:53:33.545727 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 730/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 30 15:53:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:53:33.837195 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e115e48-4d3c-4f6e-a5e2-7f8d42ff95dc None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:53:33.839694 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-1e115e48-4d3c-4f6e-a5e2-7f8d42ff95dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:53:33.962402 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e115e48-4d3c-4f6e-a5e2-7f8d42ff95dc None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:53:34.021603 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:53:34.040424 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c82df4f-143b-40f9-b365-79bda13b4fd6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f4157105f3488f807b7475008ab722', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:53:34.041033 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c82df4f-143b-40f9-b365-79bda13b4fd6 service neutron] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:53:34.041347 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c82df4f-143b-40f9-b365-79bda13b4fd6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:53:34.041626 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c82df4f-143b-40f9-b365-79bda13b4fd6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:53:34.043230 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-3c82df4f-143b-40f9-b365-79bda13b4fd6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:53:34.160145 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c82df4f-143b-40f9-b365-79bda13b4fd6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:53:34.161636 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c82df4f-143b-40f9-b365-79bda13b4fd6 service neutron] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:53:34.181252 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 731/1461] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 15:53:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:53:36.135658 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ad1245-6280-45d6-9b8c-10357e5aff7f None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:53:36.136244 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ad1245-6280-45d6-9b8c-10357e5aff7f None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:53:36.136244 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ad1245-6280-45d6-9b8c-10357e5aff7f None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:53:36.138164 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 731/1462] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 30 15:53:36 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 15:53:36.144152 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3626f7f-471d-4f56-a9cf-c39112e7e41f None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:53:36.144606 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3626f7f-471d-4f56-a9cf-c39112e7e41f None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:53:36.144721 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3626f7f-471d-4f56-a9cf-c39112e7e41f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:53:36.217208 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-b3626f7f-471d-4f56-a9cf-c39112e7e41f None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:53:36.218801 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-b3626f7f-471d-4f56-a9cf-c39112e7e41f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:53:36.263875 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:53:36.328519 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 732/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 30 15:53:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:53:36.622718 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ad1245-6280-45d6-9b8c-10357e5aff7f None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:53:36.624335 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-c4ad1245-6280-45d6-9b8c-10357e5aff7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:53:36.731872 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ad1245-6280-45d6-9b8c-10357e5aff7f None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:53:36.771783 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:53:36.817105 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb340ca5-6780-4ca8-b62a-5a81e9435b30 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f4157105f3488f807b7475008ab722', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:53:36.817752 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb340ca5-6780-4ca8-b62a-5a81e9435b30 service neutron] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:53:36.818027 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb340ca5-6780-4ca8-b62a-5a81e9435b30 service neutron] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:53:36.818299 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb340ca5-6780-4ca8-b62a-5a81e9435b30 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:53:36.819889 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-cb340ca5-6780-4ca8-b62a-5a81e9435b30 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:53:36.939811 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb340ca5-6780-4ca8-b62a-5a81e9435b30 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:53:36.941311 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb340ca5-6780-4ca8-b62a-5a81e9435b30 service neutron] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:53:36.960455 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 732/1464] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 15:53:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:53:38.873507 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71752141-acd8-45f7-a1d1-fba1b9f8e5c7 None None] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:53:38.874368 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71752141-acd8-45f7-a1d1-fba1b9f8e5c7 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:53:38.874368 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71752141-acd8-45f7-a1d1-fba1b9f8e5c7 None None] PATH_INFO: `/` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:53:38.876457 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 733/1465] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 30 15:53:38 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 15:53:38.883190 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9965c999-2ace-436b-9662-1edb7ea47611 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:53:38.883845 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9965c999-2ace-436b-9662-1edb7ea47611 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:53:38.883946 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9965c999-2ace-436b-9662-1edb7ea47611 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:53:38.961757 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-9965c999-2ace-436b-9662-1edb7ea47611 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:53:38.963215 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-9965c999-2ace-436b-9662-1edb7ea47611 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:53:38.984123 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:53:39.070380 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 733/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 30 15:53:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:53:39.355862 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71752141-acd8-45f7-a1d1-fba1b9f8e5c7 None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:53:39.358337 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-71752141-acd8-45f7-a1d1-fba1b9f8e5c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:53:39.472025 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71752141-acd8-45f7-a1d1-fba1b9f8e5c7 None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:53:39.539788 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:53:39.546648 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8794d91c-259e-47b0-b607-31fa200e57be service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f4157105f3488f807b7475008ab722', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:53:39.547254 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8794d91c-259e-47b0-b607-31fa200e57be service neutron] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:53:39.547534 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8794d91c-259e-47b0-b607-31fa200e57be service neutron] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:53:39.547801 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8794d91c-259e-47b0-b607-31fa200e57be service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:53:39.549468 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-8794d91c-259e-47b0-b607-31fa200e57be service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:53:39.670816 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8794d91c-259e-47b0-b607-31fa200e57be service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:53:39.672295 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8794d91c-259e-47b0-b607-31fa200e57be service neutron] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:53:39.691374 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 734/1467] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 15:53:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:53:41.624124 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2af865e-9a4c-4043-bcb1-96a312872e10 None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:53:41.624676 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2af865e-9a4c-4043-bcb1-96a312872e10 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:53:41.624676 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2af865e-9a4c-4043-bcb1-96a312872e10 None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:53:41.626160 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 734/1468] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 30 15:53:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 15:53:41.631956 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535673d2-1504-42ce-88b6-f28568ba623c None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:53:41.632454 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535673d2-1504-42ce-88b6-f28568ba623c None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:53:41.632454 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535673d2-1504-42ce-88b6-f28568ba623c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:53:41.707581 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-535673d2-1504-42ce-88b6-f28568ba623c None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:53:41.708796 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-535673d2-1504-42ce-88b6-f28568ba623c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:53:41.760913 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:53:41.824065 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 735/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 30 15:53:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:53:42.121712 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2af865e-9a4c-4043-bcb1-96a312872e10 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:53:42.124196 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-b2af865e-9a4c-4043-bcb1-96a312872e10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:53:42.236364 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2af865e-9a4c-4043-bcb1-96a312872e10 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:53:42.283400 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:53:42.310914 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b24ade-4e6a-417f-80c6-75211326f123 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f4157105f3488f807b7475008ab722', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:53:42.311515 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b24ade-4e6a-417f-80c6-75211326f123 service neutron] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:53:42.311787 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b24ade-4e6a-417f-80c6-75211326f123 service neutron] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:53:42.312024 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b24ade-4e6a-417f-80c6-75211326f123 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:53:42.313696 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-54b24ade-4e6a-417f-80c6-75211326f123 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:53:42.438418 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b24ade-4e6a-417f-80c6-75211326f123 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:53:42.439948 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b24ade-4e6a-417f-80c6-75211326f123 service neutron] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:53:42.466076 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 735/1470] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 15:53:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:53:51.821892 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=22 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:53:52.462059 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=66 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:54:01.880542 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d7c627-3bc2-489d-a775-9857c8468166 None None] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:54:01.881087 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d7c627-3bc2-489d-a775-9857c8468166 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:54:01.881328 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d7c627-3bc2-489d-a775-9857c8468166 None None] PATH_INFO: `/` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:54:01.883445 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 736/1471] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 30 15:54:01 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 15:54:01.892480 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc757c7-5117-4d7d-a44d-73e970df439b None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:54:01.893145 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc757c7-5117-4d7d-a44d-73e970df439b None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:54:01.893455 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc757c7-5117-4d7d-a44d-73e970df439b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:54:01.970347 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-4cc757c7-5117-4d7d-a44d-73e970df439b None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:54:01.971888 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-4cc757c7-5117-4d7d-a44d-73e970df439b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:54:02.079555 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 736/1472] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 30 15:54:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:54:02.377016 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d01a18-7786-4778-b766-3738dbdfb231 None None] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:54:02.377337 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d01a18-7786-4778-b766-3738dbdfb231 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:54:02.377634 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d01a18-7786-4778-b766-3738dbdfb231 None None] PATH_INFO: `/` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:54:02.379011 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 737/1473] 10.222.0.20 () {54 vars in 967 bytes} [Tue Aug 30 15:54:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 15:54:02.384026 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc6e252-dc2c-4ab2-8bc9-9a7de2fcb97e None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:54:02.384341 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc6e252-dc2c-4ab2-8bc9-9a7de2fcb97e None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:54:02.384609 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc6e252-dc2c-4ab2-8bc9-9a7de2fcb97e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:54:02.463189 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-efc6e252-dc2c-4ab2-8bc9-9a7de2fcb97e None None] MFA Rules not processed for user `b70e06a7d06d4199ab4fc36342ee1d68`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:54:02.464401 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-efc6e252-dc2c-4ab2-8bc9-9a7de2fcb97e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:54:02.486509 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:54:02.546731 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 737/1474] 10.222.0.20 () {58 vars in 1064 bytes} [Tue Aug 30 15:54:02 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:54:02.554502 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d01a18-7786-4778-b766-3738dbdfb231 None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:54:02.556050 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-44d01a18-7786-4778-b766-3738dbdfb231 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:54:02.670310 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d01a18-7786-4778-b766-3738dbdfb231 None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:54:02.748391 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47385eb4-4b1a-4993-9af8-38a7e1132aae service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b70e06a7d06d4199ab4fc36342ee1d68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:54:02.748998 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47385eb4-4b1a-4993-9af8-38a7e1132aae service glance] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:54:02.749297 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47385eb4-4b1a-4993-9af8-38a7e1132aae service glance] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:54:02.749566 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47385eb4-4b1a-4993-9af8-38a7e1132aae service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:54:02.751176 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-47385eb4-4b1a-4993-9af8-38a7e1132aae service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:54:02.805419 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:54:02.872849 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47385eb4-4b1a-4993-9af8-38a7e1132aae service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:54:02.875129 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47385eb4-4b1a-4993-9af8-38a7e1132aae service glance] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:54:02.895568 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 738/1475] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 15:54:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:54:04.890205 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bee508-62f5-4e7f-b1f2-ec0161a0943b None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:54:04.890794 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bee508-62f5-4e7f-b1f2-ec0161a0943b None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:54:04.890794 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bee508-62f5-4e7f-b1f2-ec0161a0943b None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:54:04.892002 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 738/1476] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 30 15:54:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 15:54:04.898445 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd2d16a-a32c-49ce-9f06-4749400f684b None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:54:04.899164 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd2d16a-a32c-49ce-9f06-4749400f684b None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:54:04.899261 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd2d16a-a32c-49ce-9f06-4749400f684b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:54:04.976564 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-bbd2d16a-a32c-49ce-9f06-4749400f684b None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:54:04.977825 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-bbd2d16a-a32c-49ce-9f06-4749400f684b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:54:05.078813 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 739/1477] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 30 15:54:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:54:05.358229 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bee508-62f5-4e7f-b1f2-ec0161a0943b None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:54:05.359730 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-c2bee508-62f5-4e7f-b1f2-ec0161a0943b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:54:05.500058 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bee508-62f5-4e7f-b1f2-ec0161a0943b None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:54:05.553314 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:54:05.610073 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29048196-8209-4087-bf1e-3a0b19f43d2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:54:05.610606 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29048196-8209-4087-bf1e-3a0b19f43d2f admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:54:05.610882 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29048196-8209-4087-bf1e-3a0b19f43d2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:54:05.611149 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29048196-8209-4087-bf1e-3a0b19f43d2f admin admin] PATH_INFO: `/v3/projects` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:54:05.612144 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29048196-8209-4087-bf1e-3a0b19f43d2f admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:54:05.613956 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: WARNING py.warnings [None req-29048196-8209-4087-bf1e-3a0b19f43d2f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 30 15:54:05.613956 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: warnings.warn(msg) Aug 30 15:54:05.613956 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: Aug 30 15:54:05.615129 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29048196-8209-4087-bf1e-3a0b19f43d2f admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:54:05.636329 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 739/1478] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 30 15:54:05 2022] GET /identity/v3/projects => generated 2553 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 15:54:07.545383 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeadd7e0-4c52-4984-b7ff-23faef870f7c None None] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:54:07.546124 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeadd7e0-4c52-4984-b7ff-23faef870f7c None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:54:07.546124 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeadd7e0-4c52-4984-b7ff-23faef870f7c None None] PATH_INFO: `/` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:54:07.547138 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 740/1479] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 30 15:54:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 15:54:07.553616 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8ed923-bfd5-4cdc-8112-b879bb63b8b5 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:54:07.554145 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8ed923-bfd5-4cdc-8112-b879bb63b8b5 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:54:07.554145 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8ed923-bfd5-4cdc-8112-b879bb63b8b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:54:07.625646 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-1c8ed923-bfd5-4cdc-8112-b879bb63b8b5 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:54:07.626876 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-1c8ed923-bfd5-4cdc-8112-b879bb63b8b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:54:07.726439 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 740/1480] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 30 15:54:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:54:08.025707 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeadd7e0-4c52-4984-b7ff-23faef870f7c None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:54:08.028317 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-aeadd7e0-4c52-4984-b7ff-23faef870f7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:54:08.040128 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:54:08.194821 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeadd7e0-4c52-4984-b7ff-23faef870f7c None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:54:08.291545 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:54:08.327422 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4474b1-d1b9-4606-a4a2-9515d89f40ee service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a1e756ae9be4a7bac1e81acf07e8f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:54:08.328031 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4474b1-d1b9-4606-a4a2-9515d89f40ee service nova] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:54:08.328307 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4474b1-d1b9-4606-a4a2-9515d89f40ee service nova] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:54:08.328580 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4474b1-d1b9-4606-a4a2-9515d89f40ee service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:54:08.330375 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-fb4474b1-d1b9-4606-a4a2-9515d89f40ee service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:54:08.427092 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4474b1-d1b9-4606-a4a2-9515d89f40ee service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:54:08.428692 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4474b1-d1b9-4606-a4a2-9515d89f40ee service nova] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:54:08.448295 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 741/1481] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 15:54:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:54:10.531376 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a99cfd-c3d9-4033-8b57-8a456c5335cb None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:54:10.532292 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a99cfd-c3d9-4033-8b57-8a456c5335cb None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:54:10.532292 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a99cfd-c3d9-4033-8b57-8a456c5335cb None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:54:10.534584 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 741/1482] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 30 15:54:10 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 15:54:10.540076 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cb4cfe-4ad4-433a-afea-511baa718dda None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:54:10.540612 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cb4cfe-4ad4-433a-afea-511baa718dda None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:54:10.540612 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cb4cfe-4ad4-433a-afea-511baa718dda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:54:10.611781 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-26cb4cfe-4ad4-433a-afea-511baa718dda None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:54:10.613363 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-26cb4cfe-4ad4-433a-afea-511baa718dda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:54:10.667542 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:54:10.722098 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 742/1483] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 30 15:54:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:54:11.004584 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a99cfd-c3d9-4033-8b57-8a456c5335cb None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:54:11.006224 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-e4a99cfd-c3d9-4033-8b57-8a456c5335cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:54:11.033183 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:54:11.118016 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a99cfd-c3d9-4033-8b57-8a456c5335cb None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:54:11.193363 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c563a783-09da-4f62-88cb-9897aaf33299 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f4157105f3488f807b7475008ab722', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:54:11.193963 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c563a783-09da-4f62-88cb-9897aaf33299 service neutron] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:54:11.194233 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c563a783-09da-4f62-88cb-9897aaf33299 service neutron] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:54:11.194498 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c563a783-09da-4f62-88cb-9897aaf33299 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:54:11.196081 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-c563a783-09da-4f62-88cb-9897aaf33299 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:54:11.276703 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:54:11.320579 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c563a783-09da-4f62-88cb-9897aaf33299 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:54:11.322732 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c563a783-09da-4f62-88cb-9897aaf33299 service neutron] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:54:11.349650 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 742/1484] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 15:54:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:54:13.306373 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac85fb2c-345d-4c12-9483-b8e5b6c49b89 None None] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:54:13.306912 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac85fb2c-345d-4c12-9483-b8e5b6c49b89 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:54:13.306912 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac85fb2c-345d-4c12-9483-b8e5b6c49b89 None None] PATH_INFO: `/` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:54:13.308865 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 743/1485] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 30 15:54:13 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 15:54:13.316969 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a37ae2-809a-4f3c-b6db-c44f9effbeb3 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:54:13.317477 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a37ae2-809a-4f3c-b6db-c44f9effbeb3 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:54:13.317588 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a37ae2-809a-4f3c-b6db-c44f9effbeb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:54:13.389559 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-a6a37ae2-809a-4f3c-b6db-c44f9effbeb3 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:54:13.390785 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-a6a37ae2-809a-4f3c-b6db-c44f9effbeb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:54:13.501154 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 743/1486] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 30 15:54:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:54:13.788781 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac85fb2c-345d-4c12-9483-b8e5b6c49b89 None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:54:13.791265 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-ac85fb2c-345d-4c12-9483-b8e5b6c49b89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:54:13.921267 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac85fb2c-345d-4c12-9483-b8e5b6c49b89 None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:54:13.997634 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:54:13.999726 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0491caa-28db-4976-9495-2647d722e79c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f4157105f3488f807b7475008ab722', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:54:14.000313 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0491caa-28db-4976-9495-2647d722e79c service neutron] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:54:14.000590 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0491caa-28db-4976-9495-2647d722e79c service neutron] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:54:14.000859 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0491caa-28db-4976-9495-2647d722e79c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:54:14.003301 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-d0491caa-28db-4976-9495-2647d722e79c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:54:14.116276 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0491caa-28db-4976-9495-2647d722e79c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:54:14.119176 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0491caa-28db-4976-9495-2647d722e79c service neutron] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:54:14.143448 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 744/1487] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 15:54:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:54:16.187000 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb40a058-c21d-4ed5-aaa0-efdfab6fd5e9 None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:54:16.187887 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb40a058-c21d-4ed5-aaa0-efdfab6fd5e9 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:54:16.187887 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb40a058-c21d-4ed5-aaa0-efdfab6fd5e9 None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:54:16.190115 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 744/1488] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 30 15:54:16 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 15:54:16.195717 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafd2448-88d3-41a7-bc31-127df3cc2586 None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:54:16.196198 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafd2448-88d3-41a7-bc31-127df3cc2586 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:54:16.196307 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafd2448-88d3-41a7-bc31-127df3cc2586 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:54:16.267099 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-fafd2448-88d3-41a7-bc31-127df3cc2586 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:54:16.268446 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-fafd2448-88d3-41a7-bc31-127df3cc2586 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:54:16.322361 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:54:16.368656 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 745/1489] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 30 15:54:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:54:16.657850 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb40a058-c21d-4ed5-aaa0-efdfab6fd5e9 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:54:16.659447 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-cb40a058-c21d-4ed5-aaa0-efdfab6fd5e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:54:16.696800 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:54:16.773197 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb40a058-c21d-4ed5-aaa0-efdfab6fd5e9 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:54:16.859428 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ddb729-6117-4927-bd88-b445455e9423 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f4157105f3488f807b7475008ab722', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:54:16.860036 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ddb729-6117-4927-bd88-b445455e9423 service neutron] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:54:16.860310 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ddb729-6117-4927-bd88-b445455e9423 service neutron] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:54:16.860574 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ddb729-6117-4927-bd88-b445455e9423 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:54:16.862569 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-49ddb729-6117-4927-bd88-b445455e9423 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:54:17.071344 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:54:17.101777 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ddb729-6117-4927-bd88-b445455e9423 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:54:17.103224 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ddb729-6117-4927-bd88-b445455e9423 service neutron] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:54:17.121817 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 745/1490] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 15:54:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 465 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:54:19.216243 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562197ca-c4cb-43b7-adef-0a80109279d2 None None] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:54:19.216938 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562197ca-c4cb-43b7-adef-0a80109279d2 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:54:19.216938 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562197ca-c4cb-43b7-adef-0a80109279d2 None None] PATH_INFO: `/` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:54:19.218206 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 746/1491] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 30 15:54:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 15:54:19.224676 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce6fee7-943f-4103-8ccb-683aa7c4a7ec None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:54:19.225518 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce6fee7-943f-4103-8ccb-683aa7c4a7ec None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:54:19.225693 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce6fee7-943f-4103-8ccb-683aa7c4a7ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:54:19.310115 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-9ce6fee7-943f-4103-8ccb-683aa7c4a7ec None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:54:19.311987 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-9ce6fee7-943f-4103-8ccb-683aa7c4a7ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:54:19.416913 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 746/1492] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 30 15:54:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:54:19.719396 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562197ca-c4cb-43b7-adef-0a80109279d2 None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:54:19.721945 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-562197ca-c4cb-43b7-adef-0a80109279d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:54:19.845207 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562197ca-c4cb-43b7-adef-0a80109279d2 None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:54:19.921269 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d387735d-1504-496a-ac12-85e5299f6c24 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f4157105f3488f807b7475008ab722', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:54:19.921789 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d387735d-1504-496a-ac12-85e5299f6c24 service neutron] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:54:19.922062 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d387735d-1504-496a-ac12-85e5299f6c24 service neutron] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:54:19.922327 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d387735d-1504-496a-ac12-85e5299f6c24 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:54:19.923892 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-d387735d-1504-496a-ac12-85e5299f6c24 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:54:19.933509 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:54:20.052012 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d387735d-1504-496a-ac12-85e5299f6c24 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:54:20.053514 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d387735d-1504-496a-ac12-85e5299f6c24 service neutron] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:54:20.072006 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 747/1493] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 15:54:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:54:29.414987 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=82 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:54:30.069864 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=50 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:55:39.230700 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627ea998-1524-466c-8139-bb3ce10b234c None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:55:39.231362 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627ea998-1524-466c-8139-bb3ce10b234c None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:55:39.231362 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627ea998-1524-466c-8139-bb3ce10b234c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:55:39.318713 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-627ea998-1524-466c-8139-bb3ce10b234c None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:55:39.319980 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-627ea998-1524-466c-8139-bb3ce10b234c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:55:39.422076 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 747/1494] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 15:55:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:55:39.430194 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fff41e-0edb-4b82-a1ee-69676267826e None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:55:39.430806 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fff41e-0edb-4b82-a1ee-69676267826e None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:55:39.430806 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fff41e-0edb-4b82-a1ee-69676267826e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:55:39.510733 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-49fff41e-0edb-4b82-a1ee-69676267826e None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:55:39.511868 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-49fff41e-0edb-4b82-a1ee-69676267826e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:55:39.615762 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 748/1495] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 15:55:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:55:39.621162 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627ea998-1524-466c-8139-bb3ce10b234c None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:55:39.622488 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-627ea998-1524-466c-8139-bb3ce10b234c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:55:39.634398 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627ea998-1524-466c-8139-bb3ce10b234c None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:55:39.700735 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:55:39.734629 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76d4986-2171-497d-8ec5-e9702c798e13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:55:39.735241 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76d4986-2171-497d-8ec5-e9702c798e13 admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:55:39.735511 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76d4986-2171-497d-8ec5-e9702c798e13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:55:39.735779 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76d4986-2171-497d-8ec5-e9702c798e13 admin admin] PATH_INFO: `/v3/domains` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:55:39.736431 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76d4986-2171-497d-8ec5-e9702c798e13 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:55:39.736734 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76d4986-2171-497d-8ec5-e9702c798e13 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:55:39.737764 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: WARNING py.warnings [None req-b76d4986-2171-497d-8ec5-e9702c798e13 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 30 15:55:39.737764 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: warnings.warn(msg) Aug 30 15:55:39.737764 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: Aug 30 15:55:39.738651 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76d4986-2171-497d-8ec5-e9702c798e13 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:55:39.750497 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 748/1496] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Aug 30 15:55:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:55:39.759647 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fff41e-0edb-4b82-a1ee-69676267826e None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:55:39.773970 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fff41e-0edb-4b82-a1ee-69676267826e None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:55:39.840105 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:55:39.878252 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2a08db-cc01-416b-a5b4-6609e6b9e73f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:55:39.878864 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2a08db-cc01-416b-a5b4-6609e6b9e73f admin admin] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:55:39.879144 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2a08db-cc01-416b-a5b4-6609e6b9e73f admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:55:39.879406 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2a08db-cc01-416b-a5b4-6609e6b9e73f admin admin] PATH_INFO: `/v3/projects` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:55:39.880044 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2a08db-cc01-416b-a5b4-6609e6b9e73f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:55:39.881828 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: WARNING py.warnings [None req-cb2a08db-cc01-416b-a5b4-6609e6b9e73f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 30 15:55:39.881828 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: warnings.warn(msg) Aug 30 15:55:39.881828 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: Aug 30 15:55:39.883025 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2a08db-cc01-416b-a5b4-6609e6b9e73f admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:55:39.945605 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 749/1497] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Aug 30 15:55:39 2022] POST /identity/v3/projects => generated 379 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 15:55:39.950090 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76d4986-2171-497d-8ec5-e9702c798e13 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:55:39.951625 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-b76d4986-2171-497d-8ec5-e9702c798e13 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:55:39.985445 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76d4986-2171-497d-8ec5-e9702c798e13 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:55:40.081419 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b604776a-e57b-4048-be46-0324dacd9c42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:55:40.081934 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b604776a-e57b-4048-be46-0324dacd9c42 admin admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:55:40.082198 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b604776a-e57b-4048-be46-0324dacd9c42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:55:40.082456 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b604776a-e57b-4048-be46-0324dacd9c42 admin admin] PATH_INFO: `/v3/users` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:55:40.083070 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b604776a-e57b-4048-be46-0324dacd9c42 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:55:40.084067 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: WARNING py.warnings [None req-b604776a-e57b-4048-be46-0324dacd9c42 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 30 15:55:40.084067 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: warnings.warn(msg) Aug 30 15:55:40.084067 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: Aug 30 15:55:40.085259 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b604776a-e57b-4048-be46-0324dacd9c42 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:55:40.139763 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 749/1498] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Aug 30 15:55:39 2022] POST /identity/v3/users => generated 290 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 15:55:40.144848 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2a08db-cc01-416b-a5b4-6609e6b9e73f admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:55:40.147338 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-cb2a08db-cc01-416b-a5b4-6609e6b9e73f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:55:40.210754 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2a08db-cc01-416b-a5b4-6609e6b9e73f admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:55:40.294078 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:55:40.310973 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4d680a-2cc2-41cd-8084-65f796691fc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:55:40.311586 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4d680a-2cc2-41cd-8084-65f796691fc8 admin admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:55:40.311861 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4d680a-2cc2-41cd-8084-65f796691fc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:55:40.312130 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4d680a-2cc2-41cd-8084-65f796691fc8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:55:40.312778 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4d680a-2cc2-41cd-8084-65f796691fc8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:55:40.313871 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: WARNING py.warnings [None req-0e4d680a-2cc2-41cd-8084-65f796691fc8 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 30 15:55:40.313871 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: warnings.warn(msg) Aug 30 15:55:40.313871 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: Aug 30 15:55:40.314782 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4d680a-2cc2-41cd-8084-65f796691fc8 admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:55:40.327883 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 750/1499] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 30 15:55:40 2022] GET /identity/v3/roles => generated 1420 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 15:55:40.332263 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b604776a-e57b-4048-be46-0324dacd9c42 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:55:40.333763 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-b604776a-e57b-4048-be46-0324dacd9c42 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:55:40.345480 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b604776a-e57b-4048-be46-0324dacd9c42 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:55:40.373157 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:55:40.443941 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c4dd66-dc70-4265-a19f-896a37505c2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:55:40.444550 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c4dd66-dc70-4265-a19f-896a37505c2d admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:55:40.444827 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c4dd66-dc70-4265-a19f-896a37505c2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:55:40.445099 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c4dd66-dc70-4265-a19f-896a37505c2d admin admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:55:40.445778 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c4dd66-dc70-4265-a19f-896a37505c2d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:55:40.446792 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: WARNING py.warnings [None req-23c4dd66-dc70-4265-a19f-896a37505c2d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 30 15:55:40.446792 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: warnings.warn(msg) Aug 30 15:55:40.446792 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: Aug 30 15:55:40.447822 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c4dd66-dc70-4265-a19f-896a37505c2d admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:55:40.460058 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 750/1500] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 30 15:55:40 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 15:55:40.464790 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4d680a-2cc2-41cd-8084-65f796691fc8 admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:55:40.466374 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-0e4d680a-2cc2-41cd-8084-65f796691fc8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:55:40.502056 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4d680a-2cc2-41cd-8084-65f796691fc8 admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:55:40.604789 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88cb7f8-679f-48f0-a281-8e5cc05d6f85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:55:40.605403 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88cb7f8-679f-48f0-a281-8e5cc05d6f85 admin admin] REQUEST_METHOD: `PUT` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:55:40.605681 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88cb7f8-679f-48f0-a281-8e5cc05d6f85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:55:40.605952 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88cb7f8-679f-48f0-a281-8e5cc05d6f85 admin admin] PATH_INFO: `/v3/projects/c37706e49325495cadd314f0357dcab7/users/5732f8eb1307470da249ac713b7bd212/roles/df8f9f844b1b439181d75a07c1a93418` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:55:40.628792 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88cb7f8-679f-48f0-a281-8e5cc05d6f85 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c37706e49325495cadd314f0357dcab7, user_id=5732f8eb1307470da249ac713b7bd212, role_id=df8f9f844b1b439181d75a07c1a93418)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:55:40.629869 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: WARNING py.warnings [None req-f88cb7f8-679f-48f0-a281-8e5cc05d6f85 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 30 15:55:40.629869 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: warnings.warn(msg) Aug 30 15:55:40.629869 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: Aug 30 15:55:40.631844 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88cb7f8-679f-48f0-a281-8e5cc05d6f85 admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:55:40.659979 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 751/1501] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 30 15:55:40 2022] PUT /identity/v3/projects/c37706e49325495cadd314f0357dcab7/users/5732f8eb1307470da249ac713b7bd212/roles/df8f9f844b1b439181d75a07c1a93418 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:55:40.673587 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a76682-14c6-48af-a42c-ef3dbef70623 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:55:40.673944 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a76682-14c6-48af-a42c-ef3dbef70623 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:55:40.674337 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a76682-14c6-48af-a42c-ef3dbef70623 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:55:40.735437 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-36a76682-14c6-48af-a42c-ef3dbef70623 None None] MFA Rules not processed for user `5732f8eb1307470da249ac713b7bd212`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:55:40.736651 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-36a76682-14c6-48af-a42c-ef3dbef70623 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:55:40.828942 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 751/1502] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 15:55:40 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:55:40.847304 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebde3fd-3313-4ba7-9e1a-4a1ab7982da6 None None] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:55:40.847585 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebde3fd-3313-4ba7-9e1a-4a1ab7982da6 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:55:40.847852 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebde3fd-3313-4ba7-9e1a-4a1ab7982da6 None None] PATH_INFO: `/` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:55:40.849189 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 752/1503] 10.222.0.20 () {54 vars in 964 bytes} [Tue Aug 30 15:55:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 15:55:40.854870 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6d1487-3d1d-483d-864e-584d50c29d1a None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:55:40.855116 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6d1487-3d1d-483d-864e-584d50c29d1a None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:55:40.855347 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6d1487-3d1d-483d-864e-584d50c29d1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:55:40.888115 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:55:40.938408 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-fb6d1487-3d1d-483d-864e-584d50c29d1a None None] MFA Rules not processed for user `0100a04821804993866ea20f159663f9`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:55:40.939642 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-fb6d1487-3d1d-483d-864e-584d50c29d1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:55:41.033111 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 752/1504] 10.222.0.20 () {58 vars in 1061 bytes} [Tue Aug 30 15:55:40 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:55:41.047683 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebde3fd-3313-4ba7-9e1a-4a1ab7982da6 None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:55:41.050148 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-bebde3fd-3313-4ba7-9e1a-4a1ab7982da6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:55:41.108585 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:55:41.163522 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebde3fd-3313-4ba7-9e1a-4a1ab7982da6 None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:55:41.243827 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4a4f39-68e8-40bb-9dd9-1490e6d207bc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0100a04821804993866ea20f159663f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:55:41.244436 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4a4f39-68e8-40bb-9dd9-1490e6d207bc service cinder] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:55:41.244709 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4a4f39-68e8-40bb-9dd9-1490e6d207bc service cinder] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:55:41.245061 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4a4f39-68e8-40bb-9dd9-1490e6d207bc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:55:41.246809 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-0b4a4f39-68e8-40bb-9dd9-1490e6d207bc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:55:41.353876 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4a4f39-68e8-40bb-9dd9-1490e6d207bc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:55:41.355354 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4a4f39-68e8-40bb-9dd9-1490e6d207bc service cinder] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:55:41.359083 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:55:41.375504 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 753/1505] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 15:55:41 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:55:41.423988 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d783857-f81c-43ca-b40a-38e26593e83d None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:55:41.424605 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d783857-f81c-43ca-b40a-38e26593e83d None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:55:41.424702 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d783857-f81c-43ca-b40a-38e26593e83d None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:55:41.426749 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 753/1506] 10.222.0.20 () {52 vars in 830 bytes} [Tue Aug 30 15:55:41 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 15:55:41.432044 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4a4f39-68e8-40bb-9dd9-1490e6d207bc service cinder] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:55:41.433359 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-0b4a4f39-68e8-40bb-9dd9-1490e6d207bc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:55:41.539365 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4a4f39-68e8-40bb-9dd9-1490e6d207bc service cinder] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:55:41.648730 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33238e3d-f86c-4af4-8d1f-db3f07a27be3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:55:41.649406 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33238e3d-f86c-4af4-8d1f-db3f07a27be3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:55:41.649686 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33238e3d-f86c-4af4-8d1f-db3f07a27be3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:55:41.649948 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33238e3d-f86c-4af4-8d1f-db3f07a27be3 admin admin] PATH_INFO: `/v3/users/5732f8eb1307470da249ac713b7bd212` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:55:41.662885 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33238e3d-f86c-4af4-8d1f-db3f07a27be3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5732f8eb1307470da249ac713b7bd212)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:55:41.663963 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: WARNING py.warnings [None req-33238e3d-f86c-4af4-8d1f-db3f07a27be3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 30 15:55:41.663963 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: warnings.warn(msg) Aug 30 15:55:41.663963 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: Aug 30 15:55:41.665182 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33238e3d-f86c-4af4-8d1f-db3f07a27be3 admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:55:41.674895 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:55:41.740591 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-33238e3d-f86c-4af4-8d1f-db3f07a27be3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5732f8eb1307470da249ac713b7bd212'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:55:41.753013 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-33238e3d-f86c-4af4-8d1f-db3f07a27be3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5732f8eb1307470da249ac713b7bd212'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:55:41.776059 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-33238e3d-f86c-4af4-8d1f-db3f07a27be3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5732f8eb1307470da249ac713b7bd212'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:55:41.790041 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-33238e3d-f86c-4af4-8d1f-db3f07a27be3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5732f8eb1307470da249ac713b7bd212'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:55:41.790492 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-33238e3d-f86c-4af4-8d1f-db3f07a27be3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5732f8eb1307470da249ac713b7bd212'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:55:41.798184 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 754/1507] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Aug 30 15:55:41 2022] DELETE /identity/v3/users/5732f8eb1307470da249ac713b7bd212 => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:55:41.802717 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de57c294-f221-4b53-81b2-802878cced68 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:55:41.803005 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de57c294-f221-4b53-81b2-802878cced68 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:55:41.803277 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de57c294-f221-4b53-81b2-802878cced68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:55:41.860862 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-de57c294-f221-4b53-81b2-802878cced68 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:55:41.862085 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-de57c294-f221-4b53-81b2-802878cced68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:55:41.945083 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:55:41.958432 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 754/1508] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 15:55:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:55:41.969212 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33238e3d-f86c-4af4-8d1f-db3f07a27be3 admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:55:41.971152 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-33238e3d-f86c-4af4-8d1f-db3f07a27be3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:55:42.088581 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33238e3d-f86c-4af4-8d1f-db3f07a27be3 admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:55:42.151688 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:55:42.174489 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c55446-cc92-4548-9ee1-de76df068d5b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f4157105f3488f807b7475008ab722', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:55:42.175099 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c55446-cc92-4548-9ee1-de76df068d5b service neutron] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:55:42.175378 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c55446-cc92-4548-9ee1-de76df068d5b service neutron] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:55:42.175648 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c55446-cc92-4548-9ee1-de76df068d5b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:55:42.177314 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-28c55446-cc92-4548-9ee1-de76df068d5b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:55:42.297571 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c55446-cc92-4548-9ee1-de76df068d5b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:55:42.298946 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c55446-cc92-4548-9ee1-de76df068d5b service neutron] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:55:42.318627 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 755/1509] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 15:55:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:55:42.758895 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de57c294-f221-4b53-81b2-802878cced68 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:55:42.760834 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-de57c294-f221-4b53-81b2-802878cced68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:55:42.774931 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de57c294-f221-4b53-81b2-802878cced68 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:55:42.876289 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed1a08c-52ea-4da1-990d-1f16414c2928 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:55:42.876834 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed1a08c-52ea-4da1-990d-1f16414c2928 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:55:42.877105 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed1a08c-52ea-4da1-990d-1f16414c2928 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:55:42.877405 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed1a08c-52ea-4da1-990d-1f16414c2928 admin admin] PATH_INFO: `/v3/projects/c37706e49325495cadd314f0357dcab7` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:55:42.887554 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed1a08c-52ea-4da1-990d-1f16414c2928 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c37706e49325495cadd314f0357dcab7)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:55:42.888587 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: WARNING py.warnings [None req-6ed1a08c-52ea-4da1-990d-1f16414c2928 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 30 15:55:42.888587 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: warnings.warn(msg) Aug 30 15:55:42.888587 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: Aug 30 15:55:42.889833 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed1a08c-52ea-4da1-990d-1f16414c2928 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:55:42.957189 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-6ed1a08c-52ea-4da1-990d-1f16414c2928 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c37706e49325495cadd314f0357dcab7', 'request_id': 'req-6ed1a08c-52ea-4da1-990d-1f16414c2928'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:55:42.967448 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-6ed1a08c-52ea-4da1-990d-1f16414c2928 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c37706e49325495cadd314f0357dcab7', 'request_id': 'req-6ed1a08c-52ea-4da1-990d-1f16414c2928'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:55:42.981341 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-6ed1a08c-52ea-4da1-990d-1f16414c2928 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c37706e49325495cadd314f0357dcab7', 'request_id': 'req-6ed1a08c-52ea-4da1-990d-1f16414c2928'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:55:43.004481 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-6ed1a08c-52ea-4da1-990d-1f16414c2928 admin admin] The token cache is being invalidate because project c37706e49325495cadd314f0357dcab7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81511) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 15:55:43.004774 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-6ed1a08c-52ea-4da1-990d-1f16414c2928 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:55:43.005357 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-6ed1a08c-52ea-4da1-990d-1f16414c2928 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:55:43.006708 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 755/1510] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Aug 30 15:55:42 2022] DELETE /identity/v3/projects/c37706e49325495cadd314f0357dcab7 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:55:52.315988 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=62 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:55:53.002468 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:57:12.280087 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452612c5-c1eb-4b42-b395-9b9e9c207f1a None None] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:57:12.280611 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452612c5-c1eb-4b42-b395-9b9e9c207f1a None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:57:12.280611 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452612c5-c1eb-4b42-b395-9b9e9c207f1a None None] PATH_INFO: `/` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:57:12.282050 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 756/1511] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 30 15:57:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 15:57:12.287264 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724675e8-f632-4a33-aed6-fb87a92d2be8 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:57:12.287744 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724675e8-f632-4a33-aed6-fb87a92d2be8 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:57:12.287744 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724675e8-f632-4a33-aed6-fb87a92d2be8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:57:12.378057 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-724675e8-f632-4a33-aed6-fb87a92d2be8 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:57:12.379287 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-724675e8-f632-4a33-aed6-fb87a92d2be8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:57:12.491782 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 756/1512] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 30 15:57:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 15:57:12.646258 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452612c5-c1eb-4b42-b395-9b9e9c207f1a None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:57:12.648704 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-452612c5-c1eb-4b42-b395-9b9e9c207f1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:57:12.779737 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452612c5-c1eb-4b42-b395-9b9e9c207f1a None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:57:12.785915 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef1ba45-adb0-4899-9fd9-7515471de802 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a1e756ae9be4a7bac1e81acf07e8f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:57:12.786544 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef1ba45-adb0-4899-9fd9-7515471de802 service nova] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:57:12.786816 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef1ba45-adb0-4899-9fd9-7515471de802 service nova] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:57:12.787081 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef1ba45-adb0-4899-9fd9-7515471de802 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:57:12.795414 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef1ba45-adb0-4899-9fd9-7515471de802 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:57:12.797001 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef1ba45-adb0-4899-9fd9-7515471de802 service nova] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:57:12.806276 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 757/1513] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 15:57:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:57:13.625240 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff395d9-bf35-48b8-9c7a-cbed214344fb None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:57:13.625843 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff395d9-bf35-48b8-9c7a-cbed214344fb None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:57:13.625843 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff395d9-bf35-48b8-9c7a-cbed214344fb None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:57:13.627157 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 757/1514] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 30 15:57:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 15:57:13.633412 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec111da2-cbef-4e6d-83e6-170e0b1cdf8f None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:57:13.634169 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec111da2-cbef-4e6d-83e6-170e0b1cdf8f None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:57:13.634372 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec111da2-cbef-4e6d-83e6-170e0b1cdf8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:57:13.657322 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-ec111da2-cbef-4e6d-83e6-170e0b1cdf8f None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:57:13.658548 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-ec111da2-cbef-4e6d-83e6-170e0b1cdf8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:57:13.667621 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 758/1515] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 30 15:57:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:57:13.822066 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff395d9-bf35-48b8-9c7a-cbed214344fb None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:57:13.844095 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff395d9-bf35-48b8-9c7a-cbed214344fb None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:57:13.850792 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acf69af-5da5-496b-9887-d0a2bc0bc339 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a1e756ae9be4a7bac1e81acf07e8f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:57:13.851401 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acf69af-5da5-496b-9887-d0a2bc0bc339 service nova] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:57:13.851681 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acf69af-5da5-496b-9887-d0a2bc0bc339 service nova] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:57:13.851952 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acf69af-5da5-496b-9887-d0a2bc0bc339 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:57:13.861039 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7acf69af-5da5-496b-9887-d0a2bc0bc339 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:57:13.862681 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7acf69af-5da5-496b-9887-d0a2bc0bc339 service nova] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:57:13.873095 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 758/1516] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 15:57:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:57:13.915911 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9490f461-61d3-48db-bd39-4a8b7de415b2 None None] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:57:13.916346 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9490f461-61d3-48db-bd39-4a8b7de415b2 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:57:13.916640 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9490f461-61d3-48db-bd39-4a8b7de415b2 None None] PATH_INFO: `/v3` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:57:13.918106 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 759/1517] 10.222.0.42 () {54 vars in 925 bytes} [Tue Aug 30 15:57:13 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:57:13.925307 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b2d167-de87-4d82-843d-3eede35fd1fb None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:57:13.925566 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b2d167-de87-4d82-843d-3eede35fd1fb None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:57:13.925841 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b2d167-de87-4d82-843d-3eede35fd1fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:57:13.994193 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-26b2d167-de87-4d82-843d-3eede35fd1fb None None] MFA Rules not processed for user `bc12c6d31eab4a17a160250adc2380c8`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:57:13.995407 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-26b2d167-de87-4d82-843d-3eede35fd1fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:57:14.014352 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:57:14.070698 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 759/1518] 10.222.0.42 () {58 vars in 1015 bytes} [Tue Aug 30 15:57:13 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:57:22.947319 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd32eb45-6d81-4e67-a94d-af2125664a3a None None] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:57:22.948175 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd32eb45-6d81-4e67-a94d-af2125664a3a None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:57:22.948175 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd32eb45-6d81-4e67-a94d-af2125664a3a None None] PATH_INFO: `/` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:57:22.950284 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 760/1519] 10.222.0.20 () {54 vars in 961 bytes} [Tue Aug 30 15:57:22 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 15:57:22.955535 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97cefa2-5059-4312-be0e-0935ecad4def None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:57:22.955997 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97cefa2-5059-4312-be0e-0935ecad4def None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:57:22.956103 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97cefa2-5059-4312-be0e-0935ecad4def None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:57:22.978172 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-c97cefa2-5059-4312-be0e-0935ecad4def None None] MFA Rules not processed for user `bc12c6d31eab4a17a160250adc2380c8`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:57:22.979349 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-c97cefa2-5059-4312-be0e-0935ecad4def None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:57:22.988557 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 760/1520] 10.222.0.20 () {58 vars in 1058 bytes} [Tue Aug 30 15:57:22 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:57:22.997084 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd32eb45-6d81-4e67-a94d-af2125664a3a None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:57:23.013173 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd32eb45-6d81-4e67-a94d-af2125664a3a None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:57:23.019637 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fbe45a-fb10-4550-8d70-f8def69f623e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc12c6d31eab4a17a160250adc2380c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:57:23.020242 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fbe45a-fb10-4550-8d70-f8def69f623e service placement] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:57:23.020523 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fbe45a-fb10-4550-8d70-f8def69f623e service placement] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:57:23.020794 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fbe45a-fb10-4550-8d70-f8def69f623e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:57:23.029485 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60fbe45a-fb10-4550-8d70-f8def69f623e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:57:23.031054 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60fbe45a-fb10-4550-8d70-f8def69f623e service placement] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:57:23.040889 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 761/1521] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 15:57:22 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:57:25.102947 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd08765-d9e4-4e34-b3a9-998faded793c None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:57:25.103537 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd08765-d9e4-4e34-b3a9-998faded793c None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:57:25.103537 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd08765-d9e4-4e34-b3a9-998faded793c None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:57:25.104762 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 761/1522] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 30 15:57:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 15:57:25.109861 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8712a9ab-3434-4cca-b762-f74b31a1bf3a None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:57:25.110320 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8712a9ab-3434-4cca-b762-f74b31a1bf3a None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:57:25.110426 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8712a9ab-3434-4cca-b762-f74b31a1bf3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:57:25.132692 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-8712a9ab-3434-4cca-b762-f74b31a1bf3a None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:57:25.133926 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-8712a9ab-3434-4cca-b762-f74b31a1bf3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:57:25.142112 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 762/1523] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 30 15:57:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:57:25.308514 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd08765-d9e4-4e34-b3a9-998faded793c None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:57:25.332353 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd08765-d9e4-4e34-b3a9-998faded793c None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:57:25.338506 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73d2b69-b80b-43bc-81f0-5617b4787ae1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a1e756ae9be4a7bac1e81acf07e8f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:57:25.339109 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73d2b69-b80b-43bc-81f0-5617b4787ae1 service nova] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:57:25.339383 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73d2b69-b80b-43bc-81f0-5617b4787ae1 service nova] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:57:25.339647 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73d2b69-b80b-43bc-81f0-5617b4787ae1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:57:25.347946 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73d2b69-b80b-43bc-81f0-5617b4787ae1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:57:25.349614 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73d2b69-b80b-43bc-81f0-5617b4787ae1 service nova] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:57:25.359254 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 762/1524] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 15:57:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:57:26.189104 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b113c2c-2fdd-41de-b950-8b6a3d939d28 None None] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:57:26.190078 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b113c2c-2fdd-41de-b950-8b6a3d939d28 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:57:26.190078 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b113c2c-2fdd-41de-b950-8b6a3d939d28 None None] PATH_INFO: `/` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:57:26.192398 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 763/1525] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 30 15:57:26 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 15:57:26.198484 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e39c6bd-a4e9-4863-97ee-4c9783f7382c None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:57:26.198955 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e39c6bd-a4e9-4863-97ee-4c9783f7382c None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:57:26.199061 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e39c6bd-a4e9-4863-97ee-4c9783f7382c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:57:26.222144 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-9e39c6bd-a4e9-4863-97ee-4c9783f7382c None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:57:26.223354 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-9e39c6bd-a4e9-4863-97ee-4c9783f7382c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:57:26.232554 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 763/1526] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 30 15:57:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:57:26.384178 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b113c2c-2fdd-41de-b950-8b6a3d939d28 None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:57:26.410980 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b113c2c-2fdd-41de-b950-8b6a3d939d28 None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:57:26.418902 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5a0877-ede2-4b74-a3b8-23f09aaf609c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a1e756ae9be4a7bac1e81acf07e8f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:57:26.419502 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5a0877-ede2-4b74-a3b8-23f09aaf609c service nova] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:57:26.419777 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5a0877-ede2-4b74-a3b8-23f09aaf609c service nova] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:57:26.420044 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5a0877-ede2-4b74-a3b8-23f09aaf609c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:57:26.428553 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5a0877-ede2-4b74-a3b8-23f09aaf609c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:57:26.430184 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5a0877-ede2-4b74-a3b8-23f09aaf609c service nova] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:57:26.439710 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 764/1527] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 15:57:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:57:30.253908 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae14d0e4-6bd4-4513-9b73-3684608e657f None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:57:30.254594 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae14d0e4-6bd4-4513-9b73-3684608e657f None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:57:30.254594 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae14d0e4-6bd4-4513-9b73-3684608e657f None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:57:30.255867 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 764/1528] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 30 15:57:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 15:57:30.262120 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8776dbe9-bc97-409e-9cb3-2d419911c397 None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:57:30.262667 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8776dbe9-bc97-409e-9cb3-2d419911c397 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:57:30.262867 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8776dbe9-bc97-409e-9cb3-2d419911c397 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:57:30.291694 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-8776dbe9-bc97-409e-9cb3-2d419911c397 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:57:30.292890 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-8776dbe9-bc97-409e-9cb3-2d419911c397 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:57:30.301126 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 765/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 30 15:57:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:57:30.587424 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae14d0e4-6bd4-4513-9b73-3684608e657f None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:57:30.589653 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-ae14d0e4-6bd4-4513-9b73-3684608e657f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:57:30.683827 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae14d0e4-6bd4-4513-9b73-3684608e657f None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:57:30.687795 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566a0620-1606-4add-a18e-952eb4586ddd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f4157105f3488f807b7475008ab722', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:57:30.688400 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566a0620-1606-4add-a18e-952eb4586ddd service neutron] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:57:30.688702 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566a0620-1606-4add-a18e-952eb4586ddd service neutron] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:57:30.688940 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566a0620-1606-4add-a18e-952eb4586ddd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:57:30.698121 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566a0620-1606-4add-a18e-952eb4586ddd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:57:30.699546 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566a0620-1606-4add-a18e-952eb4586ddd service neutron] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:57:30.708886 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 765/1530] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 15:57:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:57:32.589805 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd5b6ab-50a8-4997-b943-48cd3bf2bed9 None None] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:57:32.590686 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd5b6ab-50a8-4997-b943-48cd3bf2bed9 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:57:32.590686 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd5b6ab-50a8-4997-b943-48cd3bf2bed9 None None] PATH_INFO: `/` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:57:32.592778 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 766/1531] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 30 15:57:32 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 15:57:32.598300 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b7c522-3b69-47cb-b9e0-2925b2aa265d None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:57:32.598731 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b7c522-3b69-47cb-b9e0-2925b2aa265d None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:57:32.598845 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b7c522-3b69-47cb-b9e0-2925b2aa265d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:57:32.620386 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-33b7c522-3b69-47cb-b9e0-2925b2aa265d None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:57:32.621626 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-33b7c522-3b69-47cb-b9e0-2925b2aa265d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:57:32.630813 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 766/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 30 15:57:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:57:32.916817 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd5b6ab-50a8-4997-b943-48cd3bf2bed9 None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:57:32.933071 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd5b6ab-50a8-4997-b943-48cd3bf2bed9 None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:57:32.937753 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff201f5a-8400-4902-9f5d-b50bc75f3b3a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f4157105f3488f807b7475008ab722', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:57:32.938375 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff201f5a-8400-4902-9f5d-b50bc75f3b3a service neutron] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:57:32.938668 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff201f5a-8400-4902-9f5d-b50bc75f3b3a service neutron] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:57:32.938933 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff201f5a-8400-4902-9f5d-b50bc75f3b3a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:57:32.948293 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff201f5a-8400-4902-9f5d-b50bc75f3b3a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:57:32.949770 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff201f5a-8400-4902-9f5d-b50bc75f3b3a service neutron] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:57:32.959448 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 767/1533] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 15:57:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:57:36.714979 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f0a0d2-5fc2-4784-8ad7-c75a70630b18 None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:57:36.715681 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f0a0d2-5fc2-4784-8ad7-c75a70630b18 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:57:36.715681 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f0a0d2-5fc2-4784-8ad7-c75a70630b18 None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:57:36.716861 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 767/1534] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 30 15:57:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 15:57:36.723276 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cb0c81-ab69-4845-8df4-700fe746e5e9 None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:57:36.724024 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cb0c81-ab69-4845-8df4-700fe746e5e9 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:57:36.724175 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cb0c81-ab69-4845-8df4-700fe746e5e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:57:36.751480 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-d9cb0c81-ab69-4845-8df4-700fe746e5e9 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:57:36.752693 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-d9cb0c81-ab69-4845-8df4-700fe746e5e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:57:36.762143 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 768/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 30 15:57:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:57:37.314658 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f0a0d2-5fc2-4784-8ad7-c75a70630b18 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:57:37.316568 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-a3f0a0d2-5fc2-4784-8ad7-c75a70630b18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:57:37.386210 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:57:37.400979 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f0a0d2-5fc2-4784-8ad7-c75a70630b18 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:57:37.405161 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a62944-30c5-4ae7-9030-d42a0ca314f2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0100a04821804993866ea20f159663f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:57:37.405792 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a62944-30c5-4ae7-9030-d42a0ca314f2 service cinder] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:57:37.406074 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a62944-30c5-4ae7-9030-d42a0ca314f2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:57:37.406345 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a62944-30c5-4ae7-9030-d42a0ca314f2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:57:37.415837 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a62944-30c5-4ae7-9030-d42a0ca314f2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:57:37.417289 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a62944-30c5-4ae7-9030-d42a0ca314f2 service cinder] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:57:37.427413 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 768/1536] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 15:57:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:57:39.355035 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81075d0d-1cf9-4d20-a59e-86d0829a7957 None None] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:57:39.355877 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81075d0d-1cf9-4d20-a59e-86d0829a7957 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:57:39.355877 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81075d0d-1cf9-4d20-a59e-86d0829a7957 None None] PATH_INFO: `/` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:57:39.357985 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 769/1537] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 30 15:57:39 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 15:57:39.364441 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ac8930-350a-4081-b109-d6041610d77d None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:57:39.365581 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ac8930-350a-4081-b109-d6041610d77d None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:57:39.365581 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ac8930-350a-4081-b109-d6041610d77d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:57:39.391578 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-86ac8930-350a-4081-b109-d6041610d77d None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:57:39.392793 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-86ac8930-350a-4081-b109-d6041610d77d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:57:39.401700 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 769/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 30 15:57:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:57:39.945750 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81075d0d-1cf9-4d20-a59e-86d0829a7957 None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:57:39.948372 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-81075d0d-1cf9-4d20-a59e-86d0829a7957 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:57:39.964405 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81075d0d-1cf9-4d20-a59e-86d0829a7957 None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:57:39.968416 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb48376-575b-44d5-a5cc-2be778bdd41d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0100a04821804993866ea20f159663f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:57:39.969007 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb48376-575b-44d5-a5cc-2be778bdd41d service cinder] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:57:39.969317 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb48376-575b-44d5-a5cc-2be778bdd41d service cinder] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:57:39.969588 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb48376-575b-44d5-a5cc-2be778bdd41d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:57:39.979163 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb48376-575b-44d5-a5cc-2be778bdd41d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:57:39.980601 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb48376-575b-44d5-a5cc-2be778bdd41d service cinder] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:57:39.990434 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 770/1539] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 15:57:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:57:49.386627 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=11 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:57:49.986246 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=89 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:59:02.724007 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0057a5-1227-4adc-829d-df8fbc46f5c0 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:02.725140 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0057a5-1227-4adc-829d-df8fbc46f5c0 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:02.725672 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0057a5-1227-4adc-829d-df8fbc46f5c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:02.752270 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-ef0057a5-1227-4adc-829d-df8fbc46f5c0 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:59:02.753501 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-ef0057a5-1227-4adc-829d-df8fbc46f5c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:59:02.765519 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 770/1540] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 15:59:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:59:02.775611 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999044f2-cf87-452b-be20-a94dd83defdf None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:02.776214 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999044f2-cf87-452b-be20-a94dd83defdf None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:02.776214 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999044f2-cf87-452b-be20-a94dd83defdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:02.776712 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9a3a5b-91a6-42e0-b57a-12fa26b671e4 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:02.776995 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9a3a5b-91a6-42e0-b57a-12fa26b671e4 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:02.777303 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9a3a5b-91a6-42e0-b57a-12fa26b671e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:02.810144 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-999044f2-cf87-452b-be20-a94dd83defdf None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:59:02.811338 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-ef9a3a5b-91a6-42e0-b57a-12fa26b671e4 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:59:02.811968 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-999044f2-cf87-452b-be20-a94dd83defdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:59:02.812631 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-ef9a3a5b-91a6-42e0-b57a-12fa26b671e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:59:02.826723 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 771/1541] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 15:59:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:59:02.831542 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999044f2-cf87-452b-be20-a94dd83defdf None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:02.841112 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 771/1542] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 15:59:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 15:59:02.845334 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999044f2-cf87-452b-be20-a94dd83defdf None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:02.850889 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acc8954-bd73-4309-bf0d-cbda9a000b04 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:02.851176 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acc8954-bd73-4309-bf0d-cbda9a000b04 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:02.851437 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acc8954-bd73-4309-bf0d-cbda9a000b04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:02.858454 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6792fd21-ee31-431a-929f-6d3d4f7fcdc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:02.859179 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6792fd21-ee31-431a-929f-6d3d4f7fcdc5 admin admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:02.859465 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6792fd21-ee31-431a-929f-6d3d4f7fcdc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:02.859739 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6792fd21-ee31-431a-929f-6d3d4f7fcdc5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:02.860397 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6792fd21-ee31-431a-929f-6d3d4f7fcdc5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81512) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:59:02.860698 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6792fd21-ee31-431a-929f-6d3d4f7fcdc5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:02.861848 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: WARNING py.warnings [None req-6792fd21-ee31-431a-929f-6d3d4f7fcdc5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 30 15:59:02.861848 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: warnings.warn(msg) Aug 30 15:59:02.861848 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: Aug 30 15:59:02.862754 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6792fd21-ee31-431a-929f-6d3d4f7fcdc5 admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:02.876839 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 772/1543] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Aug 30 15:59:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 15:59:02.882551 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e719449f-da21-4201-af18-5d96ee54a216 None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:02.882842 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e719449f-da21-4201-af18-5d96ee54a216 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:02.883122 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e719449f-da21-4201-af18-5d96ee54a216 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:02.890240 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-5acc8954-bd73-4309-bf0d-cbda9a000b04 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:59:02.891442 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-5acc8954-bd73-4309-bf0d-cbda9a000b04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:59:02.917575 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-e719449f-da21-4201-af18-5d96ee54a216 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:59:02.918807 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-e719449f-da21-4201-af18-5d96ee54a216 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:59:02.923873 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 772/1544] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 15:59:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 15:59:02.925028 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38ffaf2-dad5-405a-8760-d5bcdd09339e None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:02.925277 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38ffaf2-dad5-405a-8760-d5bcdd09339e None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:02.925554 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38ffaf2-dad5-405a-8760-d5bcdd09339e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:02.934043 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 773/1545] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 15:59:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:59:02.935320 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f11551e-dce6-4914-b5cd-47c614d9a834 None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:02.935615 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f11551e-dce6-4914-b5cd-47c614d9a834 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:02.935878 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f11551e-dce6-4914-b5cd-47c614d9a834 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:02.973507 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-d38ffaf2-dad5-405a-8760-d5bcdd09339e None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:59:02.974880 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-d38ffaf2-dad5-405a-8760-d5bcdd09339e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:59:02.975813 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-4f11551e-dce6-4914-b5cd-47c614d9a834 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:59:02.977071 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-4f11551e-dce6-4914-b5cd-47c614d9a834 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:59:02.983791 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 773/1546] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 15:59:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:59:02.984573 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38ffaf2-dad5-405a-8760-d5bcdd09339e None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:02.985788 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 774/1547] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 15:59:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:59:02.986608 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f11551e-dce6-4914-b5cd-47c614d9a834 None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:02.995901 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38ffaf2-dad5-405a-8760-d5bcdd09339e None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:02.997394 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f11551e-dce6-4914-b5cd-47c614d9a834 None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:03.001607 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663baff6-cd47-4ab7-8c83-45c67e96e84f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:03.002292 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663baff6-cd47-4ab7-8c83-45c67e96e84f admin admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:03.002571 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56df4cc-0b72-4a92-b2fb-2b495424d0ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:03.002750 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663baff6-cd47-4ab7-8c83-45c67e96e84f admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:03.003029 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663baff6-cd47-4ab7-8c83-45c67e96e84f admin admin] PATH_INFO: `/v3/projects` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:03.003366 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56df4cc-0b72-4a92-b2fb-2b495424d0ca admin admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:03.003643 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56df4cc-0b72-4a92-b2fb-2b495424d0ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:03.003764 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663baff6-cd47-4ab7-8c83-45c67e96e84f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:03.003945 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56df4cc-0b72-4a92-b2fb-2b495424d0ca admin admin] PATH_INFO: `/v3/domains` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:03.004590 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56df4cc-0b72-4a92-b2fb-2b495424d0ca admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81512) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:59:03.004711 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: WARNING py.warnings [None req-663baff6-cd47-4ab7-8c83-45c67e96e84f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 30 15:59:03.004711 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: warnings.warn(msg) Aug 30 15:59:03.004711 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: Aug 30 15:59:03.004917 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56df4cc-0b72-4a92-b2fb-2b495424d0ca admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:03.006003 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663baff6-cd47-4ab7-8c83-45c67e96e84f admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:03.006546 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56df4cc-0b72-4a92-b2fb-2b495424d0ca admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:03.019110 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 775/1548] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Aug 30 15:59:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:59:03.020429 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e11bc3-61e6-45be-a45c-de7d3508b87a None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:03.020722 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e11bc3-61e6-45be-a45c-de7d3508b87a None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:03.020995 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e11bc3-61e6-45be-a45c-de7d3508b87a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:03.040732 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-87e11bc3-61e6-45be-a45c-de7d3508b87a None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:59:03.042383 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-87e11bc3-61e6-45be-a45c-de7d3508b87a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:59:03.049931 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 774/1549] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Aug 30 15:59:02 2022] POST /identity/v3/projects => generated 379 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 15:59:03.049931 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3f35a4-1925-4ebb-bcca-467db99db833 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:03.049931 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3f35a4-1925-4ebb-bcca-467db99db833 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:03.049931 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3f35a4-1925-4ebb-bcca-467db99db833 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:03.054401 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 776/1550] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 15:59:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:59:03.055705 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b5adf5-f44a-473e-b5fb-630b9aaa340b None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:03.055988 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b5adf5-f44a-473e-b5fb-630b9aaa340b None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:03.056261 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b5adf5-f44a-473e-b5fb-630b9aaa340b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:03.073758 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-8b3f35a4-1925-4ebb-bcca-467db99db833 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:59:03.075011 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-8b3f35a4-1925-4ebb-bcca-467db99db833 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:59:03.075823 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-10b5adf5-f44a-473e-b5fb-630b9aaa340b None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:59:03.076975 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-10b5adf5-f44a-473e-b5fb-630b9aaa340b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:59:03.134793 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 775/1551] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 15:59:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:59:03.141313 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 777/1552] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 15:59:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:59:03.141488 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3180c063-c1b5-4b9b-a71e-74057fc42366 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:03.141488 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3180c063-c1b5-4b9b-a71e-74057fc42366 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:03.141715 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e18353-279a-40c1-b511-86b4a50c70c1 None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:03.141820 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3180c063-c1b5-4b9b-a71e-74057fc42366 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:03.141924 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e18353-279a-40c1-b511-86b4a50c70c1 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:03.141924 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e18353-279a-40c1-b511-86b4a50c70c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:03.159353 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-3180c063-c1b5-4b9b-a71e-74057fc42366 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:59:03.159637 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-e2e18353-279a-40c1-b511-86b4a50c70c1 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:59:03.160587 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-3180c063-c1b5-4b9b-a71e-74057fc42366 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:59:03.160832 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-e2e18353-279a-40c1-b511-86b4a50c70c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:59:03.168730 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 776/1553] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 15:59:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:59:03.168990 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 778/1554] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 15:59:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:59:03.170215 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a6a3ae-e1ee-4441-baf4-532f9b72506a None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:03.170496 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a6a3ae-e1ee-4441-baf4-532f9b72506a None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:03.170765 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a6a3ae-e1ee-4441-baf4-532f9b72506a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:03.173618 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e18353-279a-40c1-b511-86b4a50c70c1 None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:03.183030 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e18353-279a-40c1-b511-86b4a50c70c1 None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:03.188773 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5debea-733e-4698-9bff-637de685c2d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:03.189466 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5debea-733e-4698-9bff-637de685c2d9 admin admin] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:03.189759 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5debea-733e-4698-9bff-637de685c2d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:03.190029 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5debea-733e-4698-9bff-637de685c2d9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:03.191111 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5debea-733e-4698-9bff-637de685c2d9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:03.193374 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5debea-733e-4698-9bff-637de685c2d9 admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:03.194046 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-c3a6a3ae-e1ee-4441-baf4-532f9b72506a None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:59:03.195260 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-c3a6a3ae-e1ee-4441-baf4-532f9b72506a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:59:03.204125 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 777/1555] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 15:59:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:59:03.209410 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a6a3ae-e1ee-4441-baf4-532f9b72506a None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:03.220049 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a6a3ae-e1ee-4441-baf4-532f9b72506a None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:03.224815 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4df4b9-a1f5-417d-8a5a-0a65ad52c822 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:03.228564 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 779/1556] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Aug 30 15:59:03 2022] POST /identity/v3/projects => generated 385 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 15:59:03.228733 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4df4b9-a1f5-417d-8a5a-0a65ad52c822 admin admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:03.228733 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4df4b9-a1f5-417d-8a5a-0a65ad52c822 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:03.228973 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5debea-733e-4698-9bff-637de685c2d9 admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:03.229069 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4df4b9-a1f5-417d-8a5a-0a65ad52c822 admin admin] PATH_INFO: `/v3/users` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:03.229069 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4df4b9-a1f5-417d-8a5a-0a65ad52c822 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:03.229069 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4df4b9-a1f5-417d-8a5a-0a65ad52c822 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:03.255931 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 778/1557] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Aug 30 15:59:03 2022] POST /identity/v3/users => generated 290 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 15:59:03.255931 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db26242-7c2c-47c6-a530-0ca3ec5efda3 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:03.255931 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db26242-7c2c-47c6-a530-0ca3ec5efda3 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:03.255931 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db26242-7c2c-47c6-a530-0ca3ec5efda3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:03.273812 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-6db26242-7c2c-47c6-a530-0ca3ec5efda3 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:59:03.274980 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-6db26242-7c2c-47c6-a530-0ca3ec5efda3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:59:03.292983 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5debea-733e-4698-9bff-637de685c2d9 admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:03.298267 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc77b594-fbac-4fd1-b526-b71d9a676ee4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:03.298878 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc77b594-fbac-4fd1-b526-b71d9a676ee4 admin admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:03.299155 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc77b594-fbac-4fd1-b526-b71d9a676ee4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:03.299420 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc77b594-fbac-4fd1-b526-b71d9a676ee4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:03.300072 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc77b594-fbac-4fd1-b526-b71d9a676ee4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81512) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:59:03.300373 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc77b594-fbac-4fd1-b526-b71d9a676ee4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:03.302075 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc77b594-fbac-4fd1-b526-b71d9a676ee4 admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:03.313671 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 780/1558] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Aug 30 15:59:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:59:03.314864 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03d0b4e-0b21-437c-b288-ae6074828da5 None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:03.315452 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03d0b4e-0b21-437c-b288-ae6074828da5 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:03.315452 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03d0b4e-0b21-437c-b288-ae6074828da5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:03.334332 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 779/1559] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 15:59:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:59:03.335185 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db26242-7c2c-47c6-a530-0ca3ec5efda3 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:03.335866 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-c03d0b4e-0b21-437c-b288-ae6074828da5 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:59:03.337103 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-c03d0b4e-0b21-437c-b288-ae6074828da5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:59:03.344375 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db26242-7c2c-47c6-a530-0ca3ec5efda3 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:03.346265 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 781/1560] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 15:59:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:59:03.348894 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdc6f3a-0b8f-47be-8fbd-38a40de37ce3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:03.349534 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdc6f3a-0b8f-47be-8fbd-38a40de37ce3 admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:03.349999 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdc6f3a-0b8f-47be-8fbd-38a40de37ce3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:03.350129 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdc6f3a-0b8f-47be-8fbd-38a40de37ce3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:03.350214 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03d0b4e-0b21-437c-b288-ae6074828da5 None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:03.350709 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bdc6f3a-0b8f-47be-8fbd-38a40de37ce3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:59:03.351006 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bdc6f3a-0b8f-47be-8fbd-38a40de37ce3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:03.352607 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bdc6f3a-0b8f-47be-8fbd-38a40de37ce3 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:03.359480 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03d0b4e-0b21-437c-b288-ae6074828da5 None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:03.361908 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:59:03.365269 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89621053-d4c8-457d-987e-2f49c4f12229 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:03.373363 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89621053-d4c8-457d-987e-2f49c4f12229 admin admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:03.373526 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 780/1561] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Aug 30 15:59:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:59:03.373642 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89621053-d4c8-457d-987e-2f49c4f12229 admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:03.373642 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89621053-d4c8-457d-987e-2f49c4f12229 admin admin] PATH_INFO: `/v3/domains` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:03.373642 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89621053-d4c8-457d-987e-2f49c4f12229 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81512) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:59:03.373642 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89621053-d4c8-457d-987e-2f49c4f12229 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:03.373934 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4554f808-425b-4f42-80e0-0598f3b3b8a2 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:03.373934 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4554f808-425b-4f42-80e0-0598f3b3b8a2 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:03.373934 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4554f808-425b-4f42-80e0-0598f3b3b8a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:03.374151 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89621053-d4c8-457d-987e-2f49c4f12229 admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:03.374151 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:59:03.384411 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 782/1562] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Aug 30 15:59:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:59:03.385261 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89621053-d4c8-457d-987e-2f49c4f12229 admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:03.396222 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-4554f808-425b-4f42-80e0-0598f3b3b8a2 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:59:03.397467 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-4554f808-425b-4f42-80e0-0598f3b3b8a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:59:03.399629 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89621053-d4c8-457d-987e-2f49c4f12229 admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:03.404405 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d57510-86a6-4fea-950d-d814175dcb57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:03.405016 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d57510-86a6-4fea-950d-d814175dcb57 admin admin] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:03.405347 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d57510-86a6-4fea-950d-d814175dcb57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:03.405599 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d57510-86a6-4fea-950d-d814175dcb57 admin admin] PATH_INFO: `/v3/users` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:03.405722 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 781/1563] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 15:59:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:59:03.406224 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d57510-86a6-4fea-950d-d814175dcb57 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:03.406389 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4554f808-425b-4f42-80e0-0598f3b3b8a2 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:03.407230 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: WARNING py.warnings [None req-59d57510-86a6-4fea-950d-d814175dcb57 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 30 15:59:03.407230 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: warnings.warn(msg) Aug 30 15:59:03.407230 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: Aug 30 15:59:03.408440 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d57510-86a6-4fea-950d-d814175dcb57 admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:03.415579 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4554f808-425b-4f42-80e0-0598f3b3b8a2 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:03.420533 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686eae85-53fc-4ade-85f7-e25f54e1f2ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:03.421122 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686eae85-53fc-4ade-85f7-e25f54e1f2ff admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:03.422253 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686eae85-53fc-4ade-85f7-e25f54e1f2ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:03.422253 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686eae85-53fc-4ade-85f7-e25f54e1f2ff admin admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:03.422520 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686eae85-53fc-4ade-85f7-e25f54e1f2ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:03.424018 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686eae85-53fc-4ade-85f7-e25f54e1f2ff admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:03.434139 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 783/1564] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Aug 30 15:59:03 2022] POST /identity/v3/users => generated 293 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 15:59:03.434944 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d57510-86a6-4fea-950d-d814175dcb57 admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:03.437508 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 782/1565] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 30 15:59:03 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 15:59:03.438389 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686eae85-53fc-4ade-85f7-e25f54e1f2ff admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:03.447353 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d57510-86a6-4fea-950d-d814175dcb57 admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:03.447852 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686eae85-53fc-4ade-85f7-e25f54e1f2ff admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:03.452180 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0830d1-8c07-4771-81d6-bb18694aaf80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:03.452506 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3d009b-150d-4f24-867c-3ccb710da051 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:03.452768 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0830d1-8c07-4771-81d6-bb18694aaf80 admin admin] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:03.453042 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0830d1-8c07-4771-81d6-bb18694aaf80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:03.453148 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3d009b-150d-4f24-867c-3ccb710da051 admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:03.453340 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0830d1-8c07-4771-81d6-bb18694aaf80 admin admin] PATH_INFO: `/v3/projects` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:03.453458 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3d009b-150d-4f24-867c-3ccb710da051 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:03.453651 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3d009b-150d-4f24-867c-3ccb710da051 admin admin] PATH_INFO: `/v3/domains` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:03.453969 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0830d1-8c07-4771-81d6-bb18694aaf80 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:03.454296 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3d009b-150d-4f24-867c-3ccb710da051 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:59:03.454597 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3d009b-150d-4f24-867c-3ccb710da051 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:03.455867 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0830d1-8c07-4771-81d6-bb18694aaf80 admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:03.456182 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3d009b-150d-4f24-867c-3ccb710da051 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:03.475667 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 783/1566] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Aug 30 15:59:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:59:03.475667 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3d009b-150d-4f24-867c-3ccb710da051 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:03.479109 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3d009b-150d-4f24-867c-3ccb710da051 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:03.487545 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c73e48-830c-4ef7-90bb-f14fadc8d15c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:03.488418 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c73e48-830c-4ef7-90bb-f14fadc8d15c admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:03.488546 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c73e48-830c-4ef7-90bb-f14fadc8d15c admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:03.488736 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c73e48-830c-4ef7-90bb-f14fadc8d15c admin admin] PATH_INFO: `/v3/domains` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:03.489764 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c73e48-830c-4ef7-90bb-f14fadc8d15c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:59:03.490065 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c73e48-830c-4ef7-90bb-f14fadc8d15c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:03.491677 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c73e48-830c-4ef7-90bb-f14fadc8d15c admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:03.496074 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 784/1567] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Aug 30 15:59:03 2022] POST /identity/v3/projects => generated 385 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 15:59:03.496074 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0830d1-8c07-4771-81d6-bb18694aaf80 admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:03.505136 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 784/1568] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Aug 30 15:59:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:59:03.506077 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c73e48-830c-4ef7-90bb-f14fadc8d15c admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:03.521454 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c73e48-830c-4ef7-90bb-f14fadc8d15c admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:03.560560 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0830d1-8c07-4771-81d6-bb18694aaf80 admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:03.565576 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b41465-0801-44e9-883e-5feb148ac81d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:03.566342 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b41465-0801-44e9-883e-5feb148ac81d admin admin] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:03.566618 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b41465-0801-44e9-883e-5feb148ac81d admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:03.566887 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b41465-0801-44e9-883e-5feb148ac81d admin admin] PATH_INFO: `/v3/projects` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:03.567590 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b41465-0801-44e9-883e-5feb148ac81d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:03.569571 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b41465-0801-44e9-883e-5feb148ac81d admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:03.583670 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781bdcc5-f6c9-4616-97fb-66fa2b6c4118 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:03.584282 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781bdcc5-f6c9-4616-97fb-66fa2b6c4118 admin admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:03.584560 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781bdcc5-f6c9-4616-97fb-66fa2b6c4118 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:03.584828 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781bdcc5-f6c9-4616-97fb-66fa2b6c4118 admin admin] PATH_INFO: `/v3/projects` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:03.585497 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781bdcc5-f6c9-4616-97fb-66fa2b6c4118 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:03.587415 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781bdcc5-f6c9-4616-97fb-66fa2b6c4118 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:03.599195 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 785/1569] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Aug 30 15:59:03 2022] POST /identity/v3/projects => generated 369 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 15:59:03.600296 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b41465-0801-44e9-883e-5feb148ac81d admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:03.622589 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 785/1570] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Aug 30 15:59:03 2022] POST /identity/v3/projects => generated 373 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 15:59:03.622589 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781bdcc5-f6c9-4616-97fb-66fa2b6c4118 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:03.667736 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b41465-0801-44e9-883e-5feb148ac81d admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:03.681865 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781bdcc5-f6c9-4616-97fb-66fa2b6c4118 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:03.687141 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafb86b2-fe88-4b39-b8b0-ae209bd7419f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:03.687759 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafb86b2-fe88-4b39-b8b0-ae209bd7419f admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:03.688037 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafb86b2-fe88-4b39-b8b0-ae209bd7419f admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:03.688303 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafb86b2-fe88-4b39-b8b0-ae209bd7419f admin admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:03.688950 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafb86b2-fe88-4b39-b8b0-ae209bd7419f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:03.690607 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafb86b2-fe88-4b39-b8b0-ae209bd7419f admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:03.703830 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 786/1571] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 30 15:59:03 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 15:59:03.703830 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafb86b2-fe88-4b39-b8b0-ae209bd7419f admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:03.711504 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafb86b2-fe88-4b39-b8b0-ae209bd7419f admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:03.714311 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:59:03.716439 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d743e0db-4887-43c2-815b-0573c8735b77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:03.717043 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d743e0db-4887-43c2-815b-0573c8735b77 admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:03.717361 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d743e0db-4887-43c2-815b-0573c8735b77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:03.717623 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d743e0db-4887-43c2-815b-0573c8735b77 admin admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:03.718286 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d743e0db-4887-43c2-815b-0573c8735b77 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:03.719929 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d743e0db-4887-43c2-815b-0573c8735b77 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:03.728051 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7482876-49b2-47d9-abe9-5a89ffaa14af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:03.728694 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7482876-49b2-47d9-abe9-5a89ffaa14af admin admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:03.728971 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7482876-49b2-47d9-abe9-5a89ffaa14af admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:03.729264 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7482876-49b2-47d9-abe9-5a89ffaa14af admin admin] PATH_INFO: `/v3/domains` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:03.729789 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 787/1572] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 30 15:59:03 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 15:59:03.730284 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7482876-49b2-47d9-abe9-5a89ffaa14af admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81512) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:59:03.730284 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7482876-49b2-47d9-abe9-5a89ffaa14af admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:03.730577 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d743e0db-4887-43c2-815b-0573c8735b77 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:03.731825 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7482876-49b2-47d9-abe9-5a89ffaa14af admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:03.740585 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d743e0db-4887-43c2-815b-0573c8735b77 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:03.744160 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 786/1573] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Aug 30 15:59:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:59:03.744948 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7482876-49b2-47d9-abe9-5a89ffaa14af admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:03.745211 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d3bc1b-05d5-4c4e-ab65-b5a442fadd3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:03.745871 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d3bc1b-05d5-4c4e-ab65-b5a442fadd3e admin admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:03.746123 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d3bc1b-05d5-4c4e-ab65-b5a442fadd3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:03.746460 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d3bc1b-05d5-4c4e-ab65-b5a442fadd3e admin admin] PATH_INFO: `/v3/projects` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:03.747101 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d3bc1b-05d5-4c4e-ab65-b5a442fadd3e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:03.748999 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d3bc1b-05d5-4c4e-ab65-b5a442fadd3e admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:03.756447 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7482876-49b2-47d9-abe9-5a89ffaa14af admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:03.761278 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23abed0-e238-4fe9-8108-bd3562744569 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:03.761873 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23abed0-e238-4fe9-8108-bd3562744569 admin admin] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:03.762179 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23abed0-e238-4fe9-8108-bd3562744569 admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:03.762451 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23abed0-e238-4fe9-8108-bd3562744569 admin admin] PATH_INFO: `/v3/users` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:03.763077 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23abed0-e238-4fe9-8108-bd3562744569 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:03.764995 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23abed0-e238-4fe9-8108-bd3562744569 admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:03.777833 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 788/1574] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Aug 30 15:59:03 2022] POST /identity/v3/projects => generated 389 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 15:59:03.778713 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d3bc1b-05d5-4c4e-ab65-b5a442fadd3e admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:03.786529 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:59:03.788015 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 787/1575] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Aug 30 15:59:03 2022] POST /identity/v3/users => generated 293 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 15:59:03.789093 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23abed0-e238-4fe9-8108-bd3562744569 admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:03.799484 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23abed0-e238-4fe9-8108-bd3562744569 admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:03.844659 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d3bc1b-05d5-4c4e-ab65-b5a442fadd3e admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:03.850421 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e4649e-c649-42ff-9340-073fdb9bc392 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:03.851039 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e4649e-c649-42ff-9340-073fdb9bc392 admin admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:03.851318 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e4649e-c649-42ff-9340-073fdb9bc392 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:03.851586 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e4649e-c649-42ff-9340-073fdb9bc392 admin admin] PATH_INFO: `/v3/projects` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:03.852542 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e4649e-c649-42ff-9340-073fdb9bc392 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:03.854241 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e4649e-c649-42ff-9340-073fdb9bc392 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:03.858065 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d68e51-6bfc-47b1-a59a-4e281b2b153b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:03.858658 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d68e51-6bfc-47b1-a59a-4e281b2b153b admin admin] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:03.858939 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d68e51-6bfc-47b1-a59a-4e281b2b153b admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:03.859210 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d68e51-6bfc-47b1-a59a-4e281b2b153b admin admin] PATH_INFO: `/v3/users` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:03.859838 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d68e51-6bfc-47b1-a59a-4e281b2b153b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:03.861794 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d68e51-6bfc-47b1-a59a-4e281b2b153b admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:03.886256 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 789/1576] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Aug 30 15:59:03 2022] POST /identity/v3/projects => generated 379 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 15:59:03.886256 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e4649e-c649-42ff-9340-073fdb9bc392 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:03.886586 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 788/1577] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Aug 30 15:59:03 2022] POST /identity/v3/users => generated 285 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 15:59:03.886586 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d68e51-6bfc-47b1-a59a-4e281b2b153b admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:03.896175 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d68e51-6bfc-47b1-a59a-4e281b2b153b admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:03.949919 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e4649e-c649-42ff-9340-073fdb9bc392 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:03.952129 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f76f76-5daf-4263-8de6-c59200ab3770 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:03.952784 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f76f76-5daf-4263-8de6-c59200ab3770 admin admin] REQUEST_METHOD: `PUT` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:03.953057 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f76f76-5daf-4263-8de6-c59200ab3770 admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:03.953353 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f76f76-5daf-4263-8de6-c59200ab3770 admin admin] PATH_INFO: `/v3/projects/972006ddc9c04f1b838860912bbabe0c/users/096f60ad096243d3829d256db17a5f8a/roles/df8f9f844b1b439181d75a07c1a93418` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:03.954488 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384b0f3c-4d2f-48fe-b0a1-fb4dd3151965 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:03.955093 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384b0f3c-4d2f-48fe-b0a1-fb4dd3151965 admin admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:03.955374 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384b0f3c-4d2f-48fe-b0a1-fb4dd3151965 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:03.955643 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384b0f3c-4d2f-48fe-b0a1-fb4dd3151965 admin admin] PATH_INFO: `/v3/users` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:03.956272 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384b0f3c-4d2f-48fe-b0a1-fb4dd3151965 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:03.965371 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384b0f3c-4d2f-48fe-b0a1-fb4dd3151965 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:03.967987 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f76f76-5daf-4263-8de6-c59200ab3770 admin admin] RBAC: Authorizing `identity:create_grant(project_id=972006ddc9c04f1b838860912bbabe0c, user_id=096f60ad096243d3829d256db17a5f8a, role_id=df8f9f844b1b439181d75a07c1a93418)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:03.970784 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f76f76-5daf-4263-8de6-c59200ab3770 admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:03.989537 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 789/1578] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 30 15:59:03 2022] PUT /identity/v3/projects/972006ddc9c04f1b838860912bbabe0c/users/096f60ad096243d3829d256db17a5f8a/roles/df8f9f844b1b439181d75a07c1a93418 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:59:03.993575 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f76f76-5daf-4263-8de6-c59200ab3770 admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:03.993709 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 790/1579] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Aug 30 15:59:03 2022] POST /identity/v3/users => generated 287 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 15:59:03.993709 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384b0f3c-4d2f-48fe-b0a1-fb4dd3151965 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:04.002551 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384b0f3c-4d2f-48fe-b0a1-fb4dd3151965 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:04.035612 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:59:04.060740 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f76f76-5daf-4263-8de6-c59200ab3770 admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:04.061204 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07b9b03-4f6e-4d52-955b-527817b320ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:04.061845 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07b9b03-4f6e-4d52-955b-527817b320ca admin admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:04.062121 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07b9b03-4f6e-4d52-955b-527817b320ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:04.062390 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07b9b03-4f6e-4d52-955b-527817b320ca admin admin] PATH_INFO: `/v3/projects` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:04.063026 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07b9b03-4f6e-4d52-955b-527817b320ca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:04.064947 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07b9b03-4f6e-4d52-955b-527817b320ca admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:04.065870 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101adbd6-81e7-40a7-aa1b-72759236f2d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:04.066543 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101adbd6-81e7-40a7-aa1b-72759236f2d2 admin admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:04.066885 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101adbd6-81e7-40a7-aa1b-72759236f2d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:04.067219 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101adbd6-81e7-40a7-aa1b-72759236f2d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:04.067936 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101adbd6-81e7-40a7-aa1b-72759236f2d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:04.069813 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101adbd6-81e7-40a7-aa1b-72759236f2d2 admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:04.081935 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 790/1580] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 30 15:59:03 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 15:59:04.082798 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101adbd6-81e7-40a7-aa1b-72759236f2d2 admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:04.092495 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101adbd6-81e7-40a7-aa1b-72759236f2d2 admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:04.097056 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 791/1581] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Aug 30 15:59:03 2022] POST /identity/v3/projects => generated 391 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 15:59:04.098023 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07b9b03-4f6e-4d52-955b-527817b320ca admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:04.103403 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:59:04.148595 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15aa002-f909-45e2-8421-53dd37571a4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:04.149337 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15aa002-f909-45e2-8421-53dd37571a4d admin admin] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:04.149620 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15aa002-f909-45e2-8421-53dd37571a4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:04.149892 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15aa002-f909-45e2-8421-53dd37571a4d admin admin] PATH_INFO: `/v3/users` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:04.150523 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15aa002-f909-45e2-8421-53dd37571a4d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:04.152464 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15aa002-f909-45e2-8421-53dd37571a4d admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:04.161703 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07b9b03-4f6e-4d52-955b-527817b320ca admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:04.166953 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae83b43-0012-4d88-a31a-23334ec01628 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:04.167614 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae83b43-0012-4d88-a31a-23334ec01628 admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:04.167960 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae83b43-0012-4d88-a31a-23334ec01628 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:04.168339 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae83b43-0012-4d88-a31a-23334ec01628 admin admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:04.168997 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae83b43-0012-4d88-a31a-23334ec01628 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:04.170696 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae83b43-0012-4d88-a31a-23334ec01628 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:04.176635 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 791/1582] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Aug 30 15:59:04 2022] POST /identity/v3/users => generated 295 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 15:59:04.179906 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15aa002-f909-45e2-8421-53dd37571a4d admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:04.180956 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 792/1583] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 30 15:59:04 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 15:59:04.181791 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae83b43-0012-4d88-a31a-23334ec01628 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:04.189010 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15aa002-f909-45e2-8421-53dd37571a4d admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:04.191748 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae83b43-0012-4d88-a31a-23334ec01628 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:04.193641 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f35f61-1d27-4dff-aadb-e99a36ff2a4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:04.194264 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f35f61-1d27-4dff-aadb-e99a36ff2a4a admin admin] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:04.194543 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f35f61-1d27-4dff-aadb-e99a36ff2a4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:04.194788 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f35f61-1d27-4dff-aadb-e99a36ff2a4a admin admin] PATH_INFO: `/v3/users` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:04.195405 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f35f61-1d27-4dff-aadb-e99a36ff2a4a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:04.197098 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a31f010-add8-4800-b031-c46809993bf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:04.197476 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f35f61-1d27-4dff-aadb-e99a36ff2a4a admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:04.197722 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a31f010-add8-4800-b031-c46809993bf0 admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:04.198000 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a31f010-add8-4800-b031-c46809993bf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:04.198270 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a31f010-add8-4800-b031-c46809993bf0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:04.198907 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a31f010-add8-4800-b031-c46809993bf0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:04.200593 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a31f010-add8-4800-b031-c46809993bf0 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:04.210476 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 793/1584] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 30 15:59:04 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 15:59:04.211732 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d894e8-c038-4fca-a236-f6d311e2f1da None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:04.212038 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d894e8-c038-4fca-a236-f6d311e2f1da None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:04.212294 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d894e8-c038-4fca-a236-f6d311e2f1da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:04.221056 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 792/1585] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Aug 30 15:59:04 2022] POST /identity/v3/users => generated 290 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 15:59:04.222320 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f35f61-1d27-4dff-aadb-e99a36ff2a4a admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:04.231217 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-12d894e8-c038-4fca-a236-f6d311e2f1da None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:59:04.232415 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-12d894e8-c038-4fca-a236-f6d311e2f1da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:59:04.233699 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f35f61-1d27-4dff-aadb-e99a36ff2a4a admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:04.238388 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96992242-f6f6-4afc-909c-4518da0da1ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:04.238982 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96992242-f6f6-4afc-909c-4518da0da1ca admin admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:04.239595 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96992242-f6f6-4afc-909c-4518da0da1ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:04.239865 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96992242-f6f6-4afc-909c-4518da0da1ca admin admin] PATH_INFO: `/v3/roles` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:04.240441 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 794/1586] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 15:59:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:59:04.240570 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96992242-f6f6-4afc-909c-4518da0da1ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:04.241179 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d894e8-c038-4fca-a236-f6d311e2f1da None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:04.242197 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96992242-f6f6-4afc-909c-4518da0da1ca admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:04.250631 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d894e8-c038-4fca-a236-f6d311e2f1da None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:04.255495 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee73237c-6acc-4717-810c-cfdb63e3734c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:04.255882 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 793/1587] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 30 15:59:04 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 15:59:04.256158 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee73237c-6acc-4717-810c-cfdb63e3734c admin admin] REQUEST_METHOD: `PUT` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:04.256439 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee73237c-6acc-4717-810c-cfdb63e3734c admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:04.256711 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee73237c-6acc-4717-810c-cfdb63e3734c admin admin] PATH_INFO: `/v3/projects/2c2aad2014bc4db49a818a6926f860d5/users/b133ae684f20468f982ae0c25020d837/roles/df8f9f844b1b439181d75a07c1a93418` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:04.257579 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96992242-f6f6-4afc-909c-4518da0da1ca admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:04.267387 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96992242-f6f6-4afc-909c-4518da0da1ca admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:04.272093 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1b0d9b-18c8-4e4e-a69d-017ad18cd06d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:04.272681 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1b0d9b-18c8-4e4e-a69d-017ad18cd06d admin admin] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:04.272955 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1b0d9b-18c8-4e4e-a69d-017ad18cd06d admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:04.273087 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee73237c-6acc-4717-810c-cfdb63e3734c admin admin] RBAC: Authorizing `identity:create_grant(project_id=2c2aad2014bc4db49a818a6926f860d5, user_id=b133ae684f20468f982ae0c25020d837, role_id=df8f9f844b1b439181d75a07c1a93418)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:04.273257 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1b0d9b-18c8-4e4e-a69d-017ad18cd06d admin admin] PATH_INFO: `/v3/users` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:04.273904 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1b0d9b-18c8-4e4e-a69d-017ad18cd06d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:04.274170 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: WARNING py.warnings [None req-ee73237c-6acc-4717-810c-cfdb63e3734c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 30 15:59:04.274170 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: warnings.warn(msg) Aug 30 15:59:04.274170 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: Aug 30 15:59:04.276155 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee73237c-6acc-4717-810c-cfdb63e3734c admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:04.276267 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1b0d9b-18c8-4e4e-a69d-017ad18cd06d admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:04.290717 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 795/1588] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 30 15:59:04 2022] PUT /identity/v3/projects/2c2aad2014bc4db49a818a6926f860d5/users/b133ae684f20468f982ae0c25020d837/roles/df8f9f844b1b439181d75a07c1a93418 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:59:04.291520 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee73237c-6acc-4717-810c-cfdb63e3734c admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:04.302069 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 794/1589] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Aug 30 15:59:04 2022] POST /identity/v3/users => generated 296 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 15:59:04.302750 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1b0d9b-18c8-4e4e-a69d-017ad18cd06d admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:04.312225 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1b0d9b-18c8-4e4e-a69d-017ad18cd06d admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:04.353972 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee73237c-6acc-4717-810c-cfdb63e3734c admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:04.360650 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82633272-cf12-4fbb-a3c3-2df47c8aa9c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:04.361397 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82633272-cf12-4fbb-a3c3-2df47c8aa9c5 admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:04.361645 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82633272-cf12-4fbb-a3c3-2df47c8aa9c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:04.361915 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82633272-cf12-4fbb-a3c3-2df47c8aa9c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:04.362592 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82633272-cf12-4fbb-a3c3-2df47c8aa9c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:04.364569 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82633272-cf12-4fbb-a3c3-2df47c8aa9c5 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:04.369912 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92749bc4-018f-4717-9572-3ae661379ca3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:04.370546 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92749bc4-018f-4717-9572-3ae661379ca3 admin admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:04.370819 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92749bc4-018f-4717-9572-3ae661379ca3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:04.371085 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92749bc4-018f-4717-9572-3ae661379ca3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:04.371862 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92749bc4-018f-4717-9572-3ae661379ca3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:04.373475 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92749bc4-018f-4717-9572-3ae661379ca3 admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:04.374627 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 796/1590] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 30 15:59:04 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 15:59:04.375503 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82633272-cf12-4fbb-a3c3-2df47c8aa9c5 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:04.383962 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 795/1591] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 30 15:59:04 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 15:59:04.384649 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92749bc4-018f-4717-9572-3ae661379ca3 admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:04.385479 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82633272-cf12-4fbb-a3c3-2df47c8aa9c5 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:04.390281 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f6569b-2f70-4410-9d41-bd6ee566ddb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:04.390893 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f6569b-2f70-4410-9d41-bd6ee566ddb8 admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:04.391156 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f6569b-2f70-4410-9d41-bd6ee566ddb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:04.391426 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f6569b-2f70-4410-9d41-bd6ee566ddb8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:04.392059 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f6569b-2f70-4410-9d41-bd6ee566ddb8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:04.393730 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f6569b-2f70-4410-9d41-bd6ee566ddb8 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:04.394432 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92749bc4-018f-4717-9572-3ae661379ca3 admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:04.399755 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1f5643-0e89-4f75-8f0e-4c50b022b3bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:04.400346 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1f5643-0e89-4f75-8f0e-4c50b022b3bf admin admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:04.400793 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1f5643-0e89-4f75-8f0e-4c50b022b3bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:04.401053 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1f5643-0e89-4f75-8f0e-4c50b022b3bf admin admin] PATH_INFO: `/v3/roles` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:04.401729 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1f5643-0e89-4f75-8f0e-4c50b022b3bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:04.403366 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1f5643-0e89-4f75-8f0e-4c50b022b3bf admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:04.403644 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 797/1592] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 30 15:59:04 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 15:59:04.404504 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f6569b-2f70-4410-9d41-bd6ee566ddb8 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:04.406232 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-b9f6569b-2f70-4410-9d41-bd6ee566ddb8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:59:04.408993 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=91,keystone:INSERT=9 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:59:04.418903 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 796/1593] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 30 15:59:04 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 15:59:04.418903 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1f5643-0e89-4f75-8f0e-4c50b022b3bf admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:04.426825 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1f5643-0e89-4f75-8f0e-4c50b022b3bf admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:04.431526 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb48132-9dae-4c98-9db3-9a6dcc7ac6ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:04.432119 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb48132-9dae-4c98-9db3-9a6dcc7ac6ce admin admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:04.432390 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb48132-9dae-4c98-9db3-9a6dcc7ac6ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:04.432655 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb48132-9dae-4c98-9db3-9a6dcc7ac6ce admin admin] PATH_INFO: `/v3/roles` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:04.433316 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb48132-9dae-4c98-9db3-9a6dcc7ac6ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:04.434988 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb48132-9dae-4c98-9db3-9a6dcc7ac6ce admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:04.444686 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 797/1594] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 30 15:59:04 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 15:59:04.445874 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f23c5af-a7f3-4141-824e-4faf4fa4621a None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:04.446156 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f23c5af-a7f3-4141-824e-4faf4fa4621a None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:04.446427 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f23c5af-a7f3-4141-824e-4faf4fa4621a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:04.456288 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:59:04.465298 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-9f23c5af-a7f3-4141-824e-4faf4fa4621a None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:59:04.466528 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-9f23c5af-a7f3-4141-824e-4faf4fa4621a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:59:04.467752 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f6569b-2f70-4410-9d41-bd6ee566ddb8 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:04.471456 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b725b6c-a2ba-4e55-9da2-e05ed646292a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f4157105f3488f807b7475008ab722', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:04.472066 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b725b6c-a2ba-4e55-9da2-e05ed646292a service neutron] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:04.472342 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b725b6c-a2ba-4e55-9da2-e05ed646292a service neutron] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:04.472611 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b725b6c-a2ba-4e55-9da2-e05ed646292a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:04.474970 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 798/1595] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 15:59:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:59:04.475747 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f23c5af-a7f3-4141-824e-4faf4fa4621a None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:04.484912 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b725b6c-a2ba-4e55-9da2-e05ed646292a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:04.485778 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f23c5af-a7f3-4141-824e-4faf4fa4621a None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:04.486410 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b725b6c-a2ba-4e55-9da2-e05ed646292a service neutron] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:04.491245 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3b0fdf-b793-4473-9816-1175e05a434a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:04.492344 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3b0fdf-b793-4473-9816-1175e05a434a admin admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:04.492619 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3b0fdf-b793-4473-9816-1175e05a434a admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:04.492889 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3b0fdf-b793-4473-9816-1175e05a434a admin admin] PATH_INFO: `/v3/roles` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:04.493542 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3b0fdf-b793-4473-9816-1175e05a434a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:04.495125 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3b0fdf-b793-4473-9816-1175e05a434a admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:04.496203 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 798/1596] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 15:59:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:59:04.497103 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b725b6c-a2ba-4e55-9da2-e05ed646292a service neutron] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:04.508473 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 799/1597] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 30 15:59:04 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 15:59:04.508622 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b725b6c-a2ba-4e55-9da2-e05ed646292a service neutron] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:04.508728 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3b0fdf-b793-4473-9816-1175e05a434a admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:04.512623 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f551c34-50d3-48a4-8ea8-5b7db2c68e77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:04.513335 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f551c34-50d3-48a4-8ea8-5b7db2c68e77 admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:04.513790 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f551c34-50d3-48a4-8ea8-5b7db2c68e77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:04.514070 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f551c34-50d3-48a4-8ea8-5b7db2c68e77 admin admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:04.514710 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f551c34-50d3-48a4-8ea8-5b7db2c68e77 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:04.516827 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f551c34-50d3-48a4-8ea8-5b7db2c68e77 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:04.519686 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3b0fdf-b793-4473-9816-1175e05a434a admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:04.524798 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51cde17-ac43-44ed-8c50-5f7237335815 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:04.525555 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51cde17-ac43-44ed-8c50-5f7237335815 admin admin] REQUEST_METHOD: `PUT` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:04.525927 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51cde17-ac43-44ed-8c50-5f7237335815 admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:04.526122 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51cde17-ac43-44ed-8c50-5f7237335815 admin admin] PATH_INFO: `/v3/projects/08146850cb014c16ada76e8252f9b149/users/64fe0a3ca5b34e3e9420215a24342974/roles/df8f9f844b1b439181d75a07c1a93418` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:04.528023 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 799/1598] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 30 15:59:04 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 15:59:04.528967 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f551c34-50d3-48a4-8ea8-5b7db2c68e77 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:04.538585 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f551c34-50d3-48a4-8ea8-5b7db2c68e77 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:04.543254 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b41b24-c2c4-4045-83a6-0ff89c048f19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:04.543922 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b41b24-c2c4-4045-83a6-0ff89c048f19 admin admin] REQUEST_METHOD: `PUT` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:04.544201 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b41b24-c2c4-4045-83a6-0ff89c048f19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:04.544469 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b41b24-c2c4-4045-83a6-0ff89c048f19 admin admin] PATH_INFO: `/v3/projects/ea970bb384dc4e248af61a61fc848945/users/bb2d2ace7bea48938bacf1cf7fbeef5a/roles/df8f9f844b1b439181d75a07c1a93418` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:04.546058 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51cde17-ac43-44ed-8c50-5f7237335815 admin admin] RBAC: Authorizing `identity:create_grant(project_id=08146850cb014c16ada76e8252f9b149, user_id=64fe0a3ca5b34e3e9420215a24342974, role_id=df8f9f844b1b439181d75a07c1a93418)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:04.548920 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51cde17-ac43-44ed-8c50-5f7237335815 admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:04.562974 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b41b24-c2c4-4045-83a6-0ff89c048f19 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ea970bb384dc4e248af61a61fc848945, user_id=bb2d2ace7bea48938bacf1cf7fbeef5a, role_id=df8f9f844b1b439181d75a07c1a93418)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:04.565686 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b41b24-c2c4-4045-83a6-0ff89c048f19 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:04.566452 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 800/1599] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 30 15:59:04 2022] PUT /identity/v3/projects/08146850cb014c16ada76e8252f9b149/users/64fe0a3ca5b34e3e9420215a24342974/roles/df8f9f844b1b439181d75a07c1a93418 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:59:04.567366 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51cde17-ac43-44ed-8c50-5f7237335815 admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:04.590376 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 800/1600] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 30 15:59:04 2022] PUT /identity/v3/projects/ea970bb384dc4e248af61a61fc848945/users/bb2d2ace7bea48938bacf1cf7fbeef5a/roles/df8f9f844b1b439181d75a07c1a93418 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:59:04.591605 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b41b24-c2c4-4045-83a6-0ff89c048f19 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:04.657344 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51cde17-ac43-44ed-8c50-5f7237335815 admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:04.660477 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b41b24-c2c4-4045-83a6-0ff89c048f19 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:04.662519 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc4b360-9a4d-4346-bd40-ceb0a6c0781f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:04.663518 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc4b360-9a4d-4346-bd40-ceb0a6c0781f admin admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:04.663635 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc4b360-9a4d-4346-bd40-ceb0a6c0781f admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:04.663840 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc4b360-9a4d-4346-bd40-ceb0a6c0781f admin admin] PATH_INFO: `/v3/roles` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:04.664666 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc4b360-9a4d-4346-bd40-ceb0a6c0781f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:04.665370 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142b9be0-4e04-433f-8c4d-9be2628766f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:04.666119 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142b9be0-4e04-433f-8c4d-9be2628766f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:04.666300 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc4b360-9a4d-4346-bd40-ceb0a6c0781f admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:04.666507 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142b9be0-4e04-433f-8c4d-9be2628766f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:04.666848 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142b9be0-4e04-433f-8c4d-9be2628766f7 admin admin] PATH_INFO: `/v3/projects/ffc7b64dc9d141e08f1bc534fe456c76/users/dc6d3706998f41db9443bcca9766ea44/roles/df8f9f844b1b439181d75a07c1a93418` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:04.679158 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 801/1601] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 30 15:59:04 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 15:59:04.680040 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc4b360-9a4d-4346-bd40-ceb0a6c0781f admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:04.685605 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142b9be0-4e04-433f-8c4d-9be2628766f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ffc7b64dc9d141e08f1bc534fe456c76, user_id=dc6d3706998f41db9443bcca9766ea44, role_id=df8f9f844b1b439181d75a07c1a93418)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:04.688661 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142b9be0-4e04-433f-8c4d-9be2628766f7 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:04.708113 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 801/1602] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 30 15:59:04 2022] PUT /identity/v3/projects/ffc7b64dc9d141e08f1bc534fe456c76/users/dc6d3706998f41db9443bcca9766ea44/roles/df8f9f844b1b439181d75a07c1a93418 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:59:04.708912 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142b9be0-4e04-433f-8c4d-9be2628766f7 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:04.748604 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc4b360-9a4d-4346-bd40-ceb0a6c0781f admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:04.800144 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142b9be0-4e04-433f-8c4d-9be2628766f7 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:04.803551 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c177b358-d216-450c-bb73-42ba69bfdae7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f4157105f3488f807b7475008ab722', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:04.804178 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c177b358-d216-450c-bb73-42ba69bfdae7 service neutron] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:04.804455 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c177b358-d216-450c-bb73-42ba69bfdae7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:04.804655 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b68240-4ee8-4582-b8e9-7258884f808b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:04.804778 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c177b358-d216-450c-bb73-42ba69bfdae7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:04.805576 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b68240-4ee8-4582-b8e9-7258884f808b admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:04.805695 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b68240-4ee8-4582-b8e9-7258884f808b admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:04.805861 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b68240-4ee8-4582-b8e9-7258884f808b admin admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:04.806535 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b68240-4ee8-4582-b8e9-7258884f808b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:04.808100 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b68240-4ee8-4582-b8e9-7258884f808b admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:04.814430 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c177b358-d216-450c-bb73-42ba69bfdae7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:04.815878 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c177b358-d216-450c-bb73-42ba69bfdae7 service neutron] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:04.821270 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 802/1603] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 30 15:59:04 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 15:59:04.821270 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b68240-4ee8-4582-b8e9-7258884f808b admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:04.828594 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 802/1604] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 15:59:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:59:04.829971 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45daab0-f312-4ccb-87bf-6cc00abb4c1b None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:04.830456 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45daab0-f312-4ccb-87bf-6cc00abb4c1b None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:04.830722 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45daab0-f312-4ccb-87bf-6cc00abb4c1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:04.831832 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b68240-4ee8-4582-b8e9-7258884f808b admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:04.837460 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4b3b67-0d3a-4eb5-a905-23eff29e24c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:04.837899 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:59:04.838308 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4b3b67-0d3a-4eb5-a905-23eff29e24c9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:04.838677 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4b3b67-0d3a-4eb5-a905-23eff29e24c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:04.839063 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4b3b67-0d3a-4eb5-a905-23eff29e24c9 admin admin] PATH_INFO: `/v3/projects/6550878193fa4088861f08552a276578/users/252dd6215082420cade91b100c73f100/roles/df8f9f844b1b439181d75a07c1a93418` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:04.860196 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4b3b67-0d3a-4eb5-a905-23eff29e24c9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6550878193fa4088861f08552a276578, user_id=252dd6215082420cade91b100c73f100, role_id=df8f9f844b1b439181d75a07c1a93418)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:04.865434 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4b3b67-0d3a-4eb5-a905-23eff29e24c9 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:04.865639 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-d45daab0-f312-4ccb-87bf-6cc00abb4c1b None None] MFA Rules not processed for user `64fe0a3ca5b34e3e9420215a24342974`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:59:04.866673 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-d45daab0-f312-4ccb-87bf-6cc00abb4c1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:59:04.887842 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 803/1605] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 30 15:59:04 2022] PUT /identity/v3/projects/6550878193fa4088861f08552a276578/users/252dd6215082420cade91b100c73f100/roles/df8f9f844b1b439181d75a07c1a93418 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:59:04.887842 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f44d82-703d-4e29-98c1-f18a0c134850 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:04.887842 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f44d82-703d-4e29-98c1-f18a0c134850 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:04.887842 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f44d82-703d-4e29-98c1-f18a0c134850 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:04.904649 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:59:04.924919 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-50f44d82-703d-4e29-98c1-f18a0c134850 None None] MFA Rules not processed for user `bb2d2ace7bea48938bacf1cf7fbeef5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:59:04.928763 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-50f44d82-703d-4e29-98c1-f18a0c134850 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:59:04.955715 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 803/1606] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 15:59:04 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:59:04.971466 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45daab0-f312-4ccb-87bf-6cc00abb4c1b None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:05.011644 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45daab0-f312-4ccb-87bf-6cc00abb4c1b None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:05.047930 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 804/1607] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 15:59:04 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:59:05.053454 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0a598f-fb3c-4ff4-b91c-8602ed327835 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:05.053741 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0a598f-fb3c-4ff4-b91c-8602ed327835 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:05.054079 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0a598f-fb3c-4ff4-b91c-8602ed327835 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:05.094772 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-fa0a598f-fb3c-4ff4-b91c-8602ed327835 None None] MFA Rules not processed for user `dc6d3706998f41db9443bcca9766ea44`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:59:05.096581 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-fa0a598f-fb3c-4ff4-b91c-8602ed327835 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:59:05.107300 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f3f10a-9471-437f-9d14-fd44a9216e83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:05.108017 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f3f10a-9471-437f-9d14-fd44a9216e83 admin admin] REQUEST_METHOD: `PUT` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:05.109578 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f3f10a-9471-437f-9d14-fd44a9216e83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:05.109866 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f3f10a-9471-437f-9d14-fd44a9216e83 admin admin] PATH_INFO: `/v3/projects/d70595d51b57440f9c3f580281a882fe/users/2a58777cde434434bd4d64cd834fd41e/roles/df8f9f844b1b439181d75a07c1a93418` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:05.136907 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f3f10a-9471-437f-9d14-fd44a9216e83 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d70595d51b57440f9c3f580281a882fe, user_id=2a58777cde434434bd4d64cd834fd41e, role_id=df8f9f844b1b439181d75a07c1a93418)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:05.141183 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f3f10a-9471-437f-9d14-fd44a9216e83 admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:05.173261 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 804/1608] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 30 15:59:04 2022] PUT /identity/v3/projects/d70595d51b57440f9c3f580281a882fe/users/2a58777cde434434bd4d64cd834fd41e/roles/df8f9f844b1b439181d75a07c1a93418 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:59:05.173261 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f3f10a-9471-437f-9d14-fd44a9216e83 admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:05.197625 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 805/1609] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 15:59:05 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:59:05.199166 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48037331-0003-4292-9ff8-b94610a54418 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:05.199479 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48037331-0003-4292-9ff8-b94610a54418 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:05.199959 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48037331-0003-4292-9ff8-b94610a54418 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:05.224616 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-48037331-0003-4292-9ff8-b94610a54418 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:59:05.227732 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-48037331-0003-4292-9ff8-b94610a54418 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:59:05.257472 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f3f10a-9471-437f-9d14-fd44a9216e83 admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:05.263556 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d40115-4104-4728-8487-e58fd170ec47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:05.264394 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d40115-4104-4728-8487-e58fd170ec47 admin admin] REQUEST_METHOD: `PUT` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:05.268496 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d40115-4104-4728-8487-e58fd170ec47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:05.268496 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d40115-4104-4728-8487-e58fd170ec47 admin admin] PATH_INFO: `/v3/projects/10a598aa5be54fc29948da53d8ac7a32/users/c34836a75f144f74aa3fd6a38ccb9791/roles/df8f9f844b1b439181d75a07c1a93418` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:05.293048 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d40115-4104-4728-8487-e58fd170ec47 admin admin] RBAC: Authorizing `identity:create_grant(project_id=10a598aa5be54fc29948da53d8ac7a32, user_id=c34836a75f144f74aa3fd6a38ccb9791, role_id=df8f9f844b1b439181d75a07c1a93418)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:05.299231 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d40115-4104-4728-8487-e58fd170ec47 admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:05.305111 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 806/1610] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 15:59:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:59:05.305597 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f627da-4567-4457-9c04-09a24aff6090 None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:05.305734 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f627da-4567-4457-9c04-09a24aff6090 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:05.305980 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f627da-4567-4457-9c04-09a24aff6090 None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:05.307494 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 807/1611] 10.222.0.20 () {54 vars in 962 bytes} [Tue Aug 30 15:59:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 15:59:05.314021 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f627da-4567-4457-9c04-09a24aff6090 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:05.319823 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:59:05.328859 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f627da-4567-4457-9c04-09a24aff6090 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:05.330191 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 805/1612] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 30 15:59:05 2022] PUT /identity/v3/projects/10a598aa5be54fc29948da53d8ac7a32/users/c34836a75f144f74aa3fd6a38ccb9791/roles/df8f9f844b1b439181d75a07c1a93418 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:59:05.333432 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60353c2-ac24-47ac-bab8-20a256ef3480 None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:05.334212 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60353c2-ac24-47ac-bab8-20a256ef3480 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:05.335233 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60353c2-ac24-47ac-bab8-20a256ef3480 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:05.371001 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:59:05.375796 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-e60353c2-ac24-47ac-bab8-20a256ef3480 None None] MFA Rules not processed for user `2a58777cde434434bd4d64cd834fd41e`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:59:05.377372 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-e60353c2-ac24-47ac-bab8-20a256ef3480 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:59:05.445375 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ced977-a5ca-49a7-b970-094b7b85731d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:05.446036 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ced977-a5ca-49a7-b970-094b7b85731d admin admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:05.446291 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ced977-a5ca-49a7-b970-094b7b85731d admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:05.446566 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ced977-a5ca-49a7-b970-094b7b85731d admin admin] PATH_INFO: `/v3/projects` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:05.447236 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ced977-a5ca-49a7-b970-094b7b85731d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:05.449279 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ced977-a5ca-49a7-b970-094b7b85731d admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:05.479139 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 806/1613] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 15:59:05 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:59:05.479139 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45abaf0-1b88-4223-8762-c202c71a261c None None] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:05.479139 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45abaf0-1b88-4223-8762-c202c71a261c None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:05.479139 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45abaf0-1b88-4223-8762-c202c71a261c None None] PATH_INFO: `/` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:05.482902 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 807/1614] 10.222.0.20 () {54 vars in 962 bytes} [Tue Aug 30 15:59:05 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 15:59:05.484791 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7feeb873-271c-44cc-a615-ab7f2f897109 None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:05.485501 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7feeb873-271c-44cc-a615-ab7f2f897109 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:05.485791 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7feeb873-271c-44cc-a615-ab7f2f897109 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:05.493065 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 808/1615] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Aug 30 15:59:05 2022] POST /identity/v3/projects => generated 367 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 15:59:05.494318 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ced977-a5ca-49a7-b970-094b7b85731d admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:05.545287 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-7feeb873-271c-44cc-a615-ab7f2f897109 None None] MFA Rules not processed for user `7a1e756ae9be4a7bac1e81acf07e8f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:59:05.546989 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-7feeb873-271c-44cc-a615-ab7f2f897109 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:59:05.570423 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ced977-a5ca-49a7-b970-094b7b85731d admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:05.575089 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6a51c8-ec2d-48d6-8b26-4eafc756d1c3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f4157105f3488f807b7475008ab722', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:05.576541 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6a51c8-ec2d-48d6-8b26-4eafc756d1c3 service neutron] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:05.576825 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6a51c8-ec2d-48d6-8b26-4eafc756d1c3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:05.577134 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6a51c8-ec2d-48d6-8b26-4eafc756d1c3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:05.644650 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 808/1616] 10.222.0.20 () {58 vars in 1059 bytes} [Tue Aug 30 15:59:05 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:59:05.646105 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f1567f-47d6-40ab-b624-acd39e4e80e5 None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:05.646396 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f1567f-47d6-40ab-b624-acd39e4e80e5 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:05.647161 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f1567f-47d6-40ab-b624-acd39e4e80e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:05.658252 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6a51c8-ec2d-48d6-8b26-4eafc756d1c3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:05.659750 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6a51c8-ec2d-48d6-8b26-4eafc756d1c3 service neutron] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:05.669463 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-20f1567f-47d6-40ab-b624-acd39e4e80e5 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:59:05.670590 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-20f1567f-47d6-40ab-b624-acd39e4e80e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:59:05.682667 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 809/1617] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 15:59:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:59:05.683545 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6a51c8-ec2d-48d6-8b26-4eafc756d1c3 service neutron] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:05.685471 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 809/1618] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 15:59:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:59:05.689664 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333d150e-53a6-4c94-8d3b-8a9e169cd54c None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:05.689949 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333d150e-53a6-4c94-8d3b-8a9e169cd54c None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:05.690222 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333d150e-53a6-4c94-8d3b-8a9e169cd54c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:05.697397 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6a51c8-ec2d-48d6-8b26-4eafc756d1c3 service neutron] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:05.706066 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d56dcb9-5922-436c-b197-baf0581f8a5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:05.706694 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d56dcb9-5922-436c-b197-baf0581f8a5e admin admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:05.707142 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d56dcb9-5922-436c-b197-baf0581f8a5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:05.707420 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d56dcb9-5922-436c-b197-baf0581f8a5e admin admin] PATH_INFO: `/v3/projects` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:05.708063 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d56dcb9-5922-436c-b197-baf0581f8a5e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:05.710201 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d56dcb9-5922-436c-b197-baf0581f8a5e admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:05.717442 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-333d150e-53a6-4c94-8d3b-8a9e169cd54c None None] MFA Rules not processed for user `7a1e756ae9be4a7bac1e81acf07e8f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:59:05.719070 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-333d150e-53a6-4c94-8d3b-8a9e169cd54c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:59:05.733956 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 810/1619] 10.222.0.20 () {58 vars in 1059 bytes} [Tue Aug 30 15:59:05 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:59:05.734788 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333d150e-53a6-4c94-8d3b-8a9e169cd54c None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:05.751307 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333d150e-53a6-4c94-8d3b-8a9e169cd54c None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:05.757168 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0349e761-ae6c-4969-b16f-2d8b99cbaaaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:05.757934 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0349e761-ae6c-4969-b16f-2d8b99cbaaaf admin admin] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:05.758253 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0349e761-ae6c-4969-b16f-2d8b99cbaaaf admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:05.758529 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0349e761-ae6c-4969-b16f-2d8b99cbaaaf admin admin] PATH_INFO: `/v3/users` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:05.759216 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0349e761-ae6c-4969-b16f-2d8b99cbaaaf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:05.761543 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0349e761-ae6c-4969-b16f-2d8b99cbaaaf admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:05.763148 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 810/1620] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Aug 30 15:59:05 2022] POST /identity/v3/projects => generated 381 bytes in 80 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 15:59:05.764329 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d56dcb9-5922-436c-b197-baf0581f8a5e admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:05.808785 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 811/1621] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Aug 30 15:59:05 2022] POST /identity/v3/users => generated 290 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 15:59:05.810016 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0349e761-ae6c-4969-b16f-2d8b99cbaaaf admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:05.830813 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:59:05.835845 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0349e761-ae6c-4969-b16f-2d8b99cbaaaf admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:05.868955 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d56dcb9-5922-436c-b197-baf0581f8a5e admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:05.876156 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4b4699-7e07-485b-a94d-771ad4f91519 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a1e756ae9be4a7bac1e81acf07e8f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:05.876841 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4b4699-7e07-485b-a94d-771ad4f91519 service nova] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:05.877124 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4b4699-7e07-485b-a94d-771ad4f91519 service nova] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:05.877421 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4b4699-7e07-485b-a94d-771ad4f91519 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:05.891864 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:59:05.917020 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fab57f4-c07b-4de9-829d-fd6e66699676 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f4157105f3488f807b7475008ab722', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:05.918041 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fab57f4-c07b-4de9-829d-fd6e66699676 service neutron] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:05.918672 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fab57f4-c07b-4de9-829d-fd6e66699676 service neutron] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:05.919623 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fab57f4-c07b-4de9-829d-fd6e66699676 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:05.992092 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4b4699-7e07-485b-a94d-771ad4f91519 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:05.998596 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4b4699-7e07-485b-a94d-771ad4f91519 service nova] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:06.011733 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 811/1622] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 15:59:05 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 248 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:59:06.012717 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4b4699-7e07-485b-a94d-771ad4f91519 service nova] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:06.027702 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4b4699-7e07-485b-a94d-771ad4f91519 service nova] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:06.036363 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c2e647-81a9-4435-80db-812f3749bfcb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a1e756ae9be4a7bac1e81acf07e8f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:06.037116 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c2e647-81a9-4435-80db-812f3749bfcb service nova] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:06.037433 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c2e647-81a9-4435-80db-812f3749bfcb service nova] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:06.037838 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c2e647-81a9-4435-80db-812f3749bfcb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:06.105352 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fab57f4-c07b-4de9-829d-fd6e66699676 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:06.106866 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fab57f4-c07b-4de9-829d-fd6e66699676 service neutron] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:06.135154 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c2e647-81a9-4435-80db-812f3749bfcb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:06.136795 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 812/1623] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 15:59:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:59:06.140587 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c2e647-81a9-4435-80db-812f3749bfcb service nova] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:06.141238 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fab57f4-c07b-4de9-829d-fd6e66699676 service neutron] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:06.153071 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 812/1624] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 15:59:06 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:59:06.154395 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c2e647-81a9-4435-80db-812f3749bfcb service nova] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:06.167219 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c2e647-81a9-4435-80db-812f3749bfcb service nova] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:06.173439 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b823c74-463c-4f94-a95f-3bfe167883bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:06.173942 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fab57f4-c07b-4de9-829d-fd6e66699676 service neutron] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:06.174090 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b823c74-463c-4f94-a95f-3bfe167883bf admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:06.174621 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b823c74-463c-4f94-a95f-3bfe167883bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:06.175060 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b823c74-463c-4f94-a95f-3bfe167883bf admin admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:06.175721 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b823c74-463c-4f94-a95f-3bfe167883bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:06.177736 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b823c74-463c-4f94-a95f-3bfe167883bf admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:06.192098 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbacb766-b886-4d21-a0f7-8eebcf12856e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:06.192734 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbacb766-b886-4d21-a0f7-8eebcf12856e admin admin] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:06.193017 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbacb766-b886-4d21-a0f7-8eebcf12856e admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:06.193938 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 813/1625] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 30 15:59:06 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 15:59:06.195115 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b823c74-463c-4f94-a95f-3bfe167883bf admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:06.196099 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbacb766-b886-4d21-a0f7-8eebcf12856e admin admin] PATH_INFO: `/v3/users` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:06.196867 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbacb766-b886-4d21-a0f7-8eebcf12856e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:06.198875 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbacb766-b886-4d21-a0f7-8eebcf12856e admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:06.206488 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b823c74-463c-4f94-a95f-3bfe167883bf admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:06.221261 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e98c41d-a6b3-4696-aa72-3ccf17aa87ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:06.222750 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e98c41d-a6b3-4696-aa72-3ccf17aa87ec admin admin] REQUEST_METHOD: `PUT` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:06.223113 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e98c41d-a6b3-4696-aa72-3ccf17aa87ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:06.223781 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e98c41d-a6b3-4696-aa72-3ccf17aa87ec admin admin] PATH_INFO: `/v3/projects/2b98d7e00362424f9f7d9044fd391a3c/users/81ea05967b734878acd2c5b253e12716/roles/f265386950a641ac8413b53eda4ae34d` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:06.249366 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e98c41d-a6b3-4696-aa72-3ccf17aa87ec admin admin] RBAC: Authorizing `identity:create_grant(project_id=2b98d7e00362424f9f7d9044fd391a3c, user_id=81ea05967b734878acd2c5b253e12716, role_id=f265386950a641ac8413b53eda4ae34d)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:06.254138 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e98c41d-a6b3-4696-aa72-3ccf17aa87ec admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:06.272173 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 813/1626] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Aug 30 15:59:06 2022] POST /identity/v3/users => generated 297 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 15:59:06.273672 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbacb766-b886-4d21-a0f7-8eebcf12856e admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:06.287750 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 814/1627] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 30 15:59:06 2022] PUT /identity/v3/projects/2b98d7e00362424f9f7d9044fd391a3c/users/81ea05967b734878acd2c5b253e12716/roles/f265386950a641ac8413b53eda4ae34d => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:59:06.289100 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e98c41d-a6b3-4696-aa72-3ccf17aa87ec admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:06.297696 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbacb766-b886-4d21-a0f7-8eebcf12856e admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:06.345053 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:59:06.373178 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e98c41d-a6b3-4696-aa72-3ccf17aa87ec admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:06.379703 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1addb3e2-bbfb-4a38-baba-39d98ea8ee3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:06.380566 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1addb3e2-bbfb-4a38-baba-39d98ea8ee3c admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:06.380919 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1addb3e2-bbfb-4a38-baba-39d98ea8ee3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:06.381284 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1addb3e2-bbfb-4a38-baba-39d98ea8ee3c admin admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:06.382194 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1addb3e2-bbfb-4a38-baba-39d98ea8ee3c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:06.384240 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1addb3e2-bbfb-4a38-baba-39d98ea8ee3c admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:06.401854 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 815/1628] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 30 15:59:06 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 15:59:06.403280 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1addb3e2-bbfb-4a38-baba-39d98ea8ee3c admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:06.416024 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1addb3e2-bbfb-4a38-baba-39d98ea8ee3c admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:06.421869 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f994a3d-b72a-463c-b44a-7e0e28949e9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:06.423399 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f994a3d-b72a-463c-b44a-7e0e28949e9d admin admin] REQUEST_METHOD: `PUT` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:06.423887 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f994a3d-b72a-463c-b44a-7e0e28949e9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:06.424289 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f994a3d-b72a-463c-b44a-7e0e28949e9d admin admin] PATH_INFO: `/v3/projects/2b98d7e00362424f9f7d9044fd391a3c/users/81ea05967b734878acd2c5b253e12716/roles/f265386950a641ac8413b53eda4ae34d` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:06.424622 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ad770e-2edd-47ae-a0ef-c22bfa108ece admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:06.425515 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ad770e-2edd-47ae-a0ef-c22bfa108ece admin admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:06.426880 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f994a3d-b72a-463c-b44a-7e0e28949e9d admin admin] RBAC: Authorizing `identity:create_grant(project_id=2b98d7e00362424f9f7d9044fd391a3c, user_id=81ea05967b734878acd2c5b253e12716, role_id=f265386950a641ac8413b53eda4ae34d)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:06.427406 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ad770e-2edd-47ae-a0ef-c22bfa108ece admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:06.427834 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ad770e-2edd-47ae-a0ef-c22bfa108ece admin admin] PATH_INFO: `/v3/roles` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:06.429089 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ad770e-2edd-47ae-a0ef-c22bfa108ece admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:06.430713 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f994a3d-b72a-463c-b44a-7e0e28949e9d admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:06.432578 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ad770e-2edd-47ae-a0ef-c22bfa108ece admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:06.460982 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 814/1629] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 30 15:59:06 2022] GET /identity/v3/roles => generated 1420 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 15:59:06.467263 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ad770e-2edd-47ae-a0ef-c22bfa108ece admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:06.476907 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 816/1630] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 30 15:59:06 2022] PUT /identity/v3/projects/2b98d7e00362424f9f7d9044fd391a3c/users/81ea05967b734878acd2c5b253e12716/roles/f265386950a641ac8413b53eda4ae34d => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:59:06.479334 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e9b15f-4992-4e0e-af75-425f38b28f50 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:06.479334 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e9b15f-4992-4e0e-af75-425f38b28f50 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:06.479762 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e9b15f-4992-4e0e-af75-425f38b28f50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:06.492967 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ad770e-2edd-47ae-a0ef-c22bfa108ece admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:06.525680 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-d7e9b15f-4992-4e0e-af75-425f38b28f50 None None] MFA Rules not processed for user `81ea05967b734878acd2c5b253e12716`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:59:06.527098 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-d7e9b15f-4992-4e0e-af75-425f38b28f50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:59:06.619430 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 817/1631] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 15:59:06 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:59:06.621388 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce80d89-b75c-49b9-a7de-9f1ad024c36d None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:06.624461 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce80d89-b75c-49b9-a7de-9f1ad024c36d None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:06.624461 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce80d89-b75c-49b9-a7de-9f1ad024c36d None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:06.624461 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 818/1632] 10.222.0.20 () {54 vars in 912 bytes} [Tue Aug 30 15:59:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 15:59:06.632983 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7586f224-ce0b-4abc-a3c6-633ea13d8334 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:06.633888 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7586f224-ce0b-4abc-a3c6-633ea13d8334 admin admin] REQUEST_METHOD: `PUT` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:06.634440 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7586f224-ce0b-4abc-a3c6-633ea13d8334 admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:06.635790 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7586f224-ce0b-4abc-a3c6-633ea13d8334 admin admin] PATH_INFO: `/v3/projects/014b38148dc343d69f56be728fa389b9/users/403edb6da7774ed28abcf65a847d5342/roles/f265386950a641ac8413b53eda4ae34d` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:06.636848 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36ea206-4d7f-43d9-953f-e2c600f7bbbd None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:06.637628 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36ea206-4d7f-43d9-953f-e2c600f7bbbd None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:06.638134 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36ea206-4d7f-43d9-953f-e2c600f7bbbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:06.656063 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:59:06.675135 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7586f224-ce0b-4abc-a3c6-633ea13d8334 admin admin] RBAC: Authorizing `identity:create_grant(project_id=014b38148dc343d69f56be728fa389b9, user_id=403edb6da7774ed28abcf65a847d5342, role_id=f265386950a641ac8413b53eda4ae34d)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:06.676277 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-f36ea206-4d7f-43d9-953f-e2c600f7bbbd None None] MFA Rules not processed for user `7a1e756ae9be4a7bac1e81acf07e8f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:59:06.677912 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-f36ea206-4d7f-43d9-953f-e2c600f7bbbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:59:06.682451 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7586f224-ce0b-4abc-a3c6-633ea13d8334 admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:06.712380 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 815/1633] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 30 15:59:06 2022] PUT /identity/v3/projects/014b38148dc343d69f56be728fa389b9/users/403edb6da7774ed28abcf65a847d5342/roles/f265386950a641ac8413b53eda4ae34d => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:59:06.713766 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7586f224-ce0b-4abc-a3c6-633ea13d8334 admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:06.816838 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 819/1634] 10.222.0.20 () {58 vars in 1009 bytes} [Tue Aug 30 15:59:06 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:59:06.819464 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36ea206-4d7f-43d9-953f-e2c600f7bbbd None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:06.834039 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36ea206-4d7f-43d9-953f-e2c600f7bbbd None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:06.838103 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7586f224-ce0b-4abc-a3c6-633ea13d8334 admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:06.849906 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e228fb5-8e95-44c5-a597-42610035f71d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0100a04821804993866ea20f159663f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:06.851256 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e228fb5-8e95-44c5-a597-42610035f71d service cinder] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:06.851256 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e228fb5-8e95-44c5-a597-42610035f71d service cinder] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:06.851256 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e228fb5-8e95-44c5-a597-42610035f71d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:06.893344 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:59:06.935389 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a836251-6e9a-441c-9aa6-2721a16dc326 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:06.936062 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a836251-6e9a-441c-9aa6-2721a16dc326 admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:06.937439 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a836251-6e9a-441c-9aa6-2721a16dc326 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:06.937590 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a836251-6e9a-441c-9aa6-2721a16dc326 admin admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:06.940853 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a836251-6e9a-441c-9aa6-2721a16dc326 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:06.943904 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a836251-6e9a-441c-9aa6-2721a16dc326 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:06.954105 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e228fb5-8e95-44c5-a597-42610035f71d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:06.955625 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e228fb5-8e95-44c5-a597-42610035f71d service cinder] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:06.960798 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 820/1635] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 30 15:59:06 2022] GET /identity/v3/roles => generated 1420 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 15:59:06.962421 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a836251-6e9a-441c-9aa6-2721a16dc326 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:06.975870 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 816/1636] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 15:59:06 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 263 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:59:06.977424 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d837021-8017-457f-a454-bfc24618ee90 None None] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:06.977804 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d837021-8017-457f-a454-bfc24618ee90 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:06.978201 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d837021-8017-457f-a454-bfc24618ee90 None None] PATH_INFO: `/` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:06.981367 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a836251-6e9a-441c-9aa6-2721a16dc326 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:06.985184 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01b31b0-6698-451b-869e-347c749bac53 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0100a04821804993866ea20f159663f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:06.985934 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01b31b0-6698-451b-869e-347c749bac53 service cinder] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:06.986239 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01b31b0-6698-451b-869e-347c749bac53 service cinder] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:06.986522 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01b31b0-6698-451b-869e-347c749bac53 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:06.987584 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 817/1637] 10.222.0.20 () {54 vars in 912 bytes} [Tue Aug 30 15:59:06 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 15:59:06.988486 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d837021-8017-457f-a454-bfc24618ee90 None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:07.016495 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d837021-8017-457f-a454-bfc24618ee90 None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:07.047109 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7e8dd9-5e62-40ee-9cfa-8eb45769ea18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:07.048203 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7e8dd9-5e62-40ee-9cfa-8eb45769ea18 admin admin] REQUEST_METHOD: `PUT` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:07.048976 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7e8dd9-5e62-40ee-9cfa-8eb45769ea18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:07.049387 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7e8dd9-5e62-40ee-9cfa-8eb45769ea18 admin admin] PATH_INFO: `/v3/projects/014b38148dc343d69f56be728fa389b9/users/403edb6da7774ed28abcf65a847d5342/roles/f265386950a641ac8413b53eda4ae34d` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:07.052267 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7e8dd9-5e62-40ee-9cfa-8eb45769ea18 admin admin] RBAC: Authorizing `identity:create_grant(project_id=014b38148dc343d69f56be728fa389b9, user_id=403edb6da7774ed28abcf65a847d5342, role_id=f265386950a641ac8413b53eda4ae34d)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:07.056562 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7e8dd9-5e62-40ee-9cfa-8eb45769ea18 admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:07.100806 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 818/1638] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 30 15:59:06 2022] PUT /identity/v3/projects/014b38148dc343d69f56be728fa389b9/users/403edb6da7774ed28abcf65a847d5342/roles/f265386950a641ac8413b53eda4ae34d => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:59:07.102462 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1ea9ed-e65f-48a1-9571-c19c189d5786 None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:07.102745 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1ea9ed-e65f-48a1-9571-c19c189d5786 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:07.103019 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1ea9ed-e65f-48a1-9571-c19c189d5786 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:07.109288 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01b31b0-6698-451b-869e-347c749bac53 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:07.110769 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01b31b0-6698-451b-869e-347c749bac53 service cinder] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:07.125049 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 821/1639] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 15:59:06 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:59:07.126031 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01b31b0-6698-451b-869e-347c749bac53 service cinder] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:07.127619 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-8f1ea9ed-e65f-48a1-9571-c19c189d5786 None None] MFA Rules not processed for user `7a1e756ae9be4a7bac1e81acf07e8f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:59:07.128766 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-8f1ea9ed-e65f-48a1-9571-c19c189d5786 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:59:07.155166 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01b31b0-6698-451b-869e-347c749bac53 service cinder] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:07.263327 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 819/1640] 10.222.0.20 () {58 vars in 1009 bytes} [Tue Aug 30 15:59:07 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:59:07.265013 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc18c5a4-60f9-45ab-a0e3-e5d325889bb8 None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:07.266487 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc18c5a4-60f9-45ab-a0e3-e5d325889bb8 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:07.266915 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc18c5a4-60f9-45ab-a0e3-e5d325889bb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:07.294057 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b5b803-8f2d-494a-88eb-efeceb195a30 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0100a04821804993866ea20f159663f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:07.294689 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b5b803-8f2d-494a-88eb-efeceb195a30 service cinder] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:07.294971 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b5b803-8f2d-494a-88eb-efeceb195a30 service cinder] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:07.296440 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b5b803-8f2d-494a-88eb-efeceb195a30 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:07.369349 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-bc18c5a4-60f9-45ab-a0e3-e5d325889bb8 None None] MFA Rules not processed for user `403edb6da7774ed28abcf65a847d5342`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:59:07.446516 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-bc18c5a4-60f9-45ab-a0e3-e5d325889bb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:59:07.446516 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:59:07.449346 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b5b803-8f2d-494a-88eb-efeceb195a30 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:07.449346 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b5b803-8f2d-494a-88eb-efeceb195a30 service cinder] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:07.460905 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 822/1641] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 15:59:07 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:59:07.589059 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 820/1642] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 15:59:07 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 320 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:59:08.131357 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b5b803-8f2d-494a-88eb-efeceb195a30 service cinder] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:08.143889 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b5b803-8f2d-494a-88eb-efeceb195a30 service cinder] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:08.158221 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bf2776-7ddb-4e09-a39a-0e13267161fd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0100a04821804993866ea20f159663f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:08.159191 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bf2776-7ddb-4e09-a39a-0e13267161fd service cinder] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:08.159473 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bf2776-7ddb-4e09-a39a-0e13267161fd service cinder] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:08.159740 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bf2776-7ddb-4e09-a39a-0e13267161fd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:08.205059 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10bf2776-7ddb-4e09-a39a-0e13267161fd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:08.206806 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10bf2776-7ddb-4e09-a39a-0e13267161fd service cinder] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:08.230746 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 823/1643] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 15:59:08 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:59:08.292613 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc18c5a4-60f9-45ab-a0e3-e5d325889bb8 None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:08.322457 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc18c5a4-60f9-45ab-a0e3-e5d325889bb8 None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:08.336627 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743fcf34-ab9d-4c9c-986f-f86d17692e1a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0100a04821804993866ea20f159663f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:08.337769 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743fcf34-ab9d-4c9c-986f-f86d17692e1a service cinder] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:08.337880 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743fcf34-ab9d-4c9c-986f-f86d17692e1a service cinder] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:08.338126 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743fcf34-ab9d-4c9c-986f-f86d17692e1a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:08.372390 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743fcf34-ab9d-4c9c-986f-f86d17692e1a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:08.376606 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743fcf34-ab9d-4c9c-986f-f86d17692e1a service cinder] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:08.412226 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 821/1644] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 15:59:08 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:59:08.852769 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bf2776-7ddb-4e09-a39a-0e13267161fd service cinder] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:08.857842 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:59:09.044507 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bf2776-7ddb-4e09-a39a-0e13267161fd service cinder] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:09.050147 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681c4a49-1f8d-43ea-9f05-32ecdf8f4bc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:09.050883 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681c4a49-1f8d-43ea-9f05-32ecdf8f4bc2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:09.051235 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681c4a49-1f8d-43ea-9f05-32ecdf8f4bc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:09.051576 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681c4a49-1f8d-43ea-9f05-32ecdf8f4bc2 admin admin] PATH_INFO: `/v3/users/dc6d3706998f41db9443bcca9766ea44` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:09.059565 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681c4a49-1f8d-43ea-9f05-32ecdf8f4bc2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dc6d3706998f41db9443bcca9766ea44)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:09.060751 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: WARNING py.warnings [None req-681c4a49-1f8d-43ea-9f05-32ecdf8f4bc2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 30 15:59:09.060751 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: warnings.warn(msg) Aug 30 15:59:09.060751 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: Aug 30 15:59:09.062220 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681c4a49-1f8d-43ea-9f05-32ecdf8f4bc2 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:09.228474 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-681c4a49-1f8d-43ea-9f05-32ecdf8f4bc2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dc6d3706998f41db9443bcca9766ea44'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:59:09.276933 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-681c4a49-1f8d-43ea-9f05-32ecdf8f4bc2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dc6d3706998f41db9443bcca9766ea44'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:59:09.313123 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-681c4a49-1f8d-43ea-9f05-32ecdf8f4bc2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dc6d3706998f41db9443bcca9766ea44'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:59:09.315644 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-681c4a49-1f8d-43ea-9f05-32ecdf8f4bc2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dc6d3706998f41db9443bcca9766ea44'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:59:09.354322 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-681c4a49-1f8d-43ea-9f05-32ecdf8f4bc2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dc6d3706998f41db9443bcca9766ea44'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:59:09.396983 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 824/1645] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Aug 30 15:59:08 2022] DELETE /identity/v3/users/dc6d3706998f41db9443bcca9766ea44 => generated 0 bytes in 545 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:59:09.402918 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c78d6f-fddd-4a21-ae9e-dc68e2916877 None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:09.402918 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c78d6f-fddd-4a21-ae9e-dc68e2916877 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:09.402918 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c78d6f-fddd-4a21-ae9e-dc68e2916877 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:09.469340 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-76c78d6f-fddd-4a21-ae9e-dc68e2916877 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:59:09.470605 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-76c78d6f-fddd-4a21-ae9e-dc68e2916877 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:59:09.590904 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681c4a49-1f8d-43ea-9f05-32ecdf8f4bc2 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:09.593761 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-681c4a49-1f8d-43ea-9f05-32ecdf8f4bc2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:59:09.684191 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 822/1646] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 15:59:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 284 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:59:09.767402 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c78d6f-fddd-4a21-ae9e-dc68e2916877 None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:09.768968 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-76c78d6f-fddd-4a21-ae9e-dc68e2916877 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:59:09.797850 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681c4a49-1f8d-43ea-9f05-32ecdf8f4bc2 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:09.810387 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507ee9e6-16a1-4aac-b5da-1a6b886c4517 tempest-VolumeQuotaClassesTest-1456620711 tempest-VolumeQuotaClassesTest-1456620711-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '403edb6da7774ed28abcf65a847d5342', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '014b38148dc343d69f56be728fa389b9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:09.811175 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507ee9e6-16a1-4aac-b5da-1a6b886c4517 tempest-VolumeQuotaClassesTest-1456620711 tempest-VolumeQuotaClassesTest-1456620711-project-admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:09.812252 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507ee9e6-16a1-4aac-b5da-1a6b886c4517 tempest-VolumeQuotaClassesTest-1456620711 tempest-VolumeQuotaClassesTest-1456620711-project-admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:09.812599 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507ee9e6-16a1-4aac-b5da-1a6b886c4517 tempest-VolumeQuotaClassesTest-1456620711 tempest-VolumeQuotaClassesTest-1456620711-project-admin] PATH_INFO: `/v3/domains` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:09.814245 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507ee9e6-16a1-4aac-b5da-1a6b886c4517 tempest-VolumeQuotaClassesTest-1456620711 tempest-VolumeQuotaClassesTest-1456620711-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:59:09.814876 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507ee9e6-16a1-4aac-b5da-1a6b886c4517 tempest-VolumeQuotaClassesTest-1456620711 tempest-VolumeQuotaClassesTest-1456620711-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:09.820800 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507ee9e6-16a1-4aac-b5da-1a6b886c4517 tempest-VolumeQuotaClassesTest-1456620711 tempest-VolumeQuotaClassesTest-1456620711-project-admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:09.878917 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 825/1647] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Aug 30 15:59:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:59:09.880147 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507ee9e6-16a1-4aac-b5da-1a6b886c4517 tempest-VolumeQuotaClassesTest-1456620711 tempest-VolumeQuotaClassesTest-1456620711-project-admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:09.880630 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c78d6f-fddd-4a21-ae9e-dc68e2916877 None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:09.889687 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c516dc05-47c9-46af-b38d-5c93508a0ac0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f4157105f3488f807b7475008ab722', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:09.890949 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c516dc05-47c9-46af-b38d-5c93508a0ac0 service neutron] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:09.893452 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c516dc05-47c9-46af-b38d-5c93508a0ac0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:09.895061 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c516dc05-47c9-46af-b38d-5c93508a0ac0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:09.910712 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507ee9e6-16a1-4aac-b5da-1a6b886c4517 tempest-VolumeQuotaClassesTest-1456620711 tempest-VolumeQuotaClassesTest-1456620711-project-admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:09.926564 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb09541-c357-4ffc-8936-d82e39e16844 tempest-VolumeQuotaClassesTest-1456620711 tempest-VolumeQuotaClassesTest-1456620711-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '403edb6da7774ed28abcf65a847d5342', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '014b38148dc343d69f56be728fa389b9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:09.927465 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c516dc05-47c9-46af-b38d-5c93508a0ac0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:09.933282 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb09541-c357-4ffc-8936-d82e39e16844 tempest-VolumeQuotaClassesTest-1456620711 tempest-VolumeQuotaClassesTest-1456620711-project-admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:09.933577 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb09541-c357-4ffc-8936-d82e39e16844 tempest-VolumeQuotaClassesTest-1456620711 tempest-VolumeQuotaClassesTest-1456620711-project-admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:09.934515 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c516dc05-47c9-46af-b38d-5c93508a0ac0 service neutron] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:09.935678 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb09541-c357-4ffc-8936-d82e39e16844 tempest-VolumeQuotaClassesTest-1456620711 tempest-VolumeQuotaClassesTest-1456620711-project-admin] PATH_INFO: `/v3/projects` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:09.936366 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb09541-c357-4ffc-8936-d82e39e16844 tempest-VolumeQuotaClassesTest-1456620711 tempest-VolumeQuotaClassesTest-1456620711-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:09.942080 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb09541-c357-4ffc-8936-d82e39e16844 tempest-VolumeQuotaClassesTest-1456620711 tempest-VolumeQuotaClassesTest-1456620711-project-admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:09.959916 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 823/1648] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 15:59:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:59:09.969581 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:59:10.000421 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 826/1649] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Aug 30 15:59:09 2022] POST /identity/v3/projects => generated 355 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 15:59:10.005327 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c516dc05-47c9-46af-b38d-5c93508a0ac0 service neutron] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:10.023142 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c516dc05-47c9-46af-b38d-5c93508a0ac0 service neutron] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:10.112534 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:59:10.181301 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ef3ecd-7f7d-4871-9cbc-3fb98fc12480 tempest-VolumeQuotaClassesTest-1456620711 tempest-VolumeQuotaClassesTest-1456620711-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '403edb6da7774ed28abcf65a847d5342', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '014b38148dc343d69f56be728fa389b9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:10.182625 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ef3ecd-7f7d-4871-9cbc-3fb98fc12480 tempest-VolumeQuotaClassesTest-1456620711 tempest-VolumeQuotaClassesTest-1456620711-project-admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:10.183112 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ef3ecd-7f7d-4871-9cbc-3fb98fc12480 tempest-VolumeQuotaClassesTest-1456620711 tempest-VolumeQuotaClassesTest-1456620711-project-admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:10.183532 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ef3ecd-7f7d-4871-9cbc-3fb98fc12480 tempest-VolumeQuotaClassesTest-1456620711 tempest-VolumeQuotaClassesTest-1456620711-project-admin] PATH_INFO: `/v3/domains` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:10.186250 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ef3ecd-7f7d-4871-9cbc-3fb98fc12480 tempest-VolumeQuotaClassesTest-1456620711 tempest-VolumeQuotaClassesTest-1456620711-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81512) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:59:10.187471 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ef3ecd-7f7d-4871-9cbc-3fb98fc12480 tempest-VolumeQuotaClassesTest-1456620711 tempest-VolumeQuotaClassesTest-1456620711-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:10.188803 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ef3ecd-7f7d-4871-9cbc-3fb98fc12480 tempest-VolumeQuotaClassesTest-1456620711 tempest-VolumeQuotaClassesTest-1456620711-project-admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:10.229716 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 824/1650] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Aug 30 15:59:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:59:10.287259 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb09541-c357-4ffc-8936-d82e39e16844 tempest-VolumeQuotaClassesTest-1456620711 tempest-VolumeQuotaClassesTest-1456620711-project-admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:10.302772 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb09541-c357-4ffc-8936-d82e39e16844 tempest-VolumeQuotaClassesTest-1456620711 tempest-VolumeQuotaClassesTest-1456620711-project-admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:10.313978 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1655f8-5f05-4cff-95f6-12c093732ab8 tempest-VolumeQuotaClassesTest-1456620711 tempest-VolumeQuotaClassesTest-1456620711-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '403edb6da7774ed28abcf65a847d5342', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '014b38148dc343d69f56be728fa389b9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:10.314643 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1655f8-5f05-4cff-95f6-12c093732ab8 tempest-VolumeQuotaClassesTest-1456620711 tempest-VolumeQuotaClassesTest-1456620711-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:10.314921 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1655f8-5f05-4cff-95f6-12c093732ab8 tempest-VolumeQuotaClassesTest-1456620711 tempest-VolumeQuotaClassesTest-1456620711-project-admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:10.315193 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1655f8-5f05-4cff-95f6-12c093732ab8 tempest-VolumeQuotaClassesTest-1456620711 tempest-VolumeQuotaClassesTest-1456620711-project-admin] PATH_INFO: `/v3/projects/81a3ece9047141e38e14015bc2d06300` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:10.319149 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1655f8-5f05-4cff-95f6-12c093732ab8 tempest-VolumeQuotaClassesTest-1456620711 tempest-VolumeQuotaClassesTest-1456620711-project-admin] RBAC: Authorizing `identity:delete_project(project_id=81a3ece9047141e38e14015bc2d06300)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:10.322681 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1655f8-5f05-4cff-95f6-12c093732ab8 tempest-VolumeQuotaClassesTest-1456620711 tempest-VolumeQuotaClassesTest-1456620711-project-admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:10.443262 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-db1655f8-5f05-4cff-95f6-12c093732ab8 tempest-VolumeQuotaClassesTest-1456620711 tempest-VolumeQuotaClassesTest-1456620711-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '81a3ece9047141e38e14015bc2d06300', 'request_id': 'req-db1655f8-5f05-4cff-95f6-12c093732ab8'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:59:10.460300 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-db1655f8-5f05-4cff-95f6-12c093732ab8 tempest-VolumeQuotaClassesTest-1456620711 tempest-VolumeQuotaClassesTest-1456620711-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '81a3ece9047141e38e14015bc2d06300', 'request_id': 'req-db1655f8-5f05-4cff-95f6-12c093732ab8'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:59:10.485536 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-db1655f8-5f05-4cff-95f6-12c093732ab8 tempest-VolumeQuotaClassesTest-1456620711 tempest-VolumeQuotaClassesTest-1456620711-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '81a3ece9047141e38e14015bc2d06300', 'request_id': 'req-db1655f8-5f05-4cff-95f6-12c093732ab8'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:59:10.508406 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-db1655f8-5f05-4cff-95f6-12c093732ab8 tempest-VolumeQuotaClassesTest-1456620711 tempest-VolumeQuotaClassesTest-1456620711-project-admin] The token cache is being invalidate because project 81a3ece9047141e38e14015bc2d06300 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81511) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 15:59:10.509085 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-db1655f8-5f05-4cff-95f6-12c093732ab8 tempest-VolumeQuotaClassesTest-1456620711 tempest-VolumeQuotaClassesTest-1456620711-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:59:10.514517 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-db1655f8-5f05-4cff-95f6-12c093732ab8 tempest-VolumeQuotaClassesTest-1456620711 tempest-VolumeQuotaClassesTest-1456620711-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:59:10.536245 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 827/1651] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Aug 30 15:59:10 2022] DELETE /identity/v3/projects/81a3ece9047141e38e14015bc2d06300 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:59:11.061004 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ef3ecd-7f7d-4871-9cbc-3fb98fc12480 tempest-VolumeQuotaClassesTest-1456620711 tempest-VolumeQuotaClassesTest-1456620711-project-admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:11.062858 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-07ef3ecd-7f7d-4871-9cbc-3fb98fc12480 tempest-VolumeQuotaClassesTest-1456620711 tempest-VolumeQuotaClassesTest-1456620711-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:59:11.245172 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ef3ecd-7f7d-4871-9cbc-3fb98fc12480 tempest-VolumeQuotaClassesTest-1456620711 tempest-VolumeQuotaClassesTest-1456620711-project-admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:11.389271 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e664fb9c-37d9-4487-9c2d-97cef75d4d92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:11.390069 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e664fb9c-37d9-4487-9c2d-97cef75d4d92 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:11.390403 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e664fb9c-37d9-4487-9c2d-97cef75d4d92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:11.390740 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e664fb9c-37d9-4487-9c2d-97cef75d4d92 admin admin] PATH_INFO: `/v3/users/2a58777cde434434bd4d64cd834fd41e` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:11.402471 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e664fb9c-37d9-4487-9c2d-97cef75d4d92 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2a58777cde434434bd4d64cd834fd41e)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:11.404778 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e664fb9c-37d9-4487-9c2d-97cef75d4d92 admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:11.468944 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1655f8-5f05-4cff-95f6-12c093732ab8 tempest-VolumeQuotaClassesTest-1456620711 tempest-VolumeQuotaClassesTest-1456620711-project-admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:11.471750 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-db1655f8-5f05-4cff-95f6-12c093732ab8 tempest-VolumeQuotaClassesTest-1456620711 tempest-VolumeQuotaClassesTest-1456620711-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:59:11.498046 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1655f8-5f05-4cff-95f6-12c093732ab8 tempest-VolumeQuotaClassesTest-1456620711 tempest-VolumeQuotaClassesTest-1456620711-project-admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:11.515570 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3920a5da-60cb-4347-9c9d-e50cb0ab8a00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:11.517247 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3920a5da-60cb-4347-9c9d-e50cb0ab8a00 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:11.517602 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-e664fb9c-37d9-4487-9c2d-97cef75d4d92 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2a58777cde434434bd4d64cd834fd41e'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:59:11.517921 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3920a5da-60cb-4347-9c9d-e50cb0ab8a00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:11.518367 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3920a5da-60cb-4347-9c9d-e50cb0ab8a00 admin admin] PATH_INFO: `/v3/projects/ffc7b64dc9d141e08f1bc534fe456c76` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:11.525161 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3920a5da-60cb-4347-9c9d-e50cb0ab8a00 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ffc7b64dc9d141e08f1bc534fe456c76)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:11.530983 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3920a5da-60cb-4347-9c9d-e50cb0ab8a00 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:11.532660 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-e664fb9c-37d9-4487-9c2d-97cef75d4d92 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2a58777cde434434bd4d64cd834fd41e'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:59:11.552903 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-e664fb9c-37d9-4487-9c2d-97cef75d4d92 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2a58777cde434434bd4d64cd834fd41e'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:59:11.562881 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:59:11.569757 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-e664fb9c-37d9-4487-9c2d-97cef75d4d92 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2a58777cde434434bd4d64cd834fd41e'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:59:11.570714 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-e664fb9c-37d9-4487-9c2d-97cef75d4d92 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2a58777cde434434bd4d64cd834fd41e'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:59:11.584988 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 825/1652] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Aug 30 15:59:11 2022] DELETE /identity/v3/users/2a58777cde434434bd4d64cd834fd41e => generated 0 bytes in 525 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:59:11.590044 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1393eea-fd11-4e6d-8704-225d3bdb5ed9 None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:11.590575 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1393eea-fd11-4e6d-8704-225d3bdb5ed9 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:11.590925 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1393eea-fd11-4e6d-8704-225d3bdb5ed9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:11.631331 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-e1393eea-fd11-4e6d-8704-225d3bdb5ed9 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:59:11.631331 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-e1393eea-fd11-4e6d-8704-225d3bdb5ed9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:59:11.745964 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-3920a5da-60cb-4347-9c9d-e50cb0ab8a00 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ffc7b64dc9d141e08f1bc534fe456c76', 'request_id': 'req-3920a5da-60cb-4347-9c9d-e50cb0ab8a00'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:59:11.787566 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-3920a5da-60cb-4347-9c9d-e50cb0ab8a00 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ffc7b64dc9d141e08f1bc534fe456c76', 'request_id': 'req-3920a5da-60cb-4347-9c9d-e50cb0ab8a00'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:59:11.803272 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-3920a5da-60cb-4347-9c9d-e50cb0ab8a00 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ffc7b64dc9d141e08f1bc534fe456c76', 'request_id': 'req-3920a5da-60cb-4347-9c9d-e50cb0ab8a00'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:59:11.848557 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-3920a5da-60cb-4347-9c9d-e50cb0ab8a00 admin admin] The token cache is being invalidate because project ffc7b64dc9d141e08f1bc534fe456c76 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81511) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 15:59:11.849008 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-3920a5da-60cb-4347-9c9d-e50cb0ab8a00 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:59:11.853906 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-3920a5da-60cb-4347-9c9d-e50cb0ab8a00 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:59:11.863528 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 828/1653] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Aug 30 15:59:11 2022] DELETE /identity/v3/projects/ffc7b64dc9d141e08f1bc534fe456c76 => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:59:11.874078 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c559bd0-fecb-4096-8267-45ad7b27853f None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:11.874078 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c559bd0-fecb-4096-8267-45ad7b27853f None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:11.874078 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c559bd0-fecb-4096-8267-45ad7b27853f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:11.888422 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 826/1654] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 15:59:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 299 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 15:59:11.899895 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1393eea-fd11-4e6d-8704-225d3bdb5ed9 None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:11.901789 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-e1393eea-fd11-4e6d-8704-225d3bdb5ed9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:59:11.929478 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-3c559bd0-fecb-4096-8267-45ad7b27853f None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:59:11.930754 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-3c559bd0-fecb-4096-8267-45ad7b27853f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:59:12.029056 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1393eea-fd11-4e6d-8704-225d3bdb5ed9 None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:12.039525 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:INSERT=3,keystone:SELECT=83,keystone:DELETE=14 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:59:12.068978 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 829/1655] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 15:59:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 15:59:12.084582 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2553eb4-ac70-43b8-aed2-60cef66ef3a1 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:12.085124 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2553eb4-ac70-43b8-aed2-60cef66ef3a1 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:12.086315 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2553eb4-ac70-43b8-aed2-60cef66ef3a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:12.163431 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-d2553eb4-ac70-43b8-aed2-60cef66ef3a1 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:59:12.165324 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-d2553eb4-ac70-43b8-aed2-60cef66ef3a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:59:12.195065 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 830/1656] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 15:59:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:59:12.196178 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2553eb4-ac70-43b8-aed2-60cef66ef3a1 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:12.218707 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2553eb4-ac70-43b8-aed2-60cef66ef3a1 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:12.234794 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d359640a-51e9-42c7-aee9-28035681e2ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f4157105f3488f807b7475008ab722', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:12.235443 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d359640a-51e9-42c7-aee9-28035681e2ba service neutron] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:12.235723 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d359640a-51e9-42c7-aee9-28035681e2ba service neutron] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:12.237332 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d359640a-51e9-42c7-aee9-28035681e2ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:12.243043 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebf6f1c-ee94-4002-8560-64c5a0cada01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:12.243963 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebf6f1c-ee94-4002-8560-64c5a0cada01 admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:12.244446 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebf6f1c-ee94-4002-8560-64c5a0cada01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:12.244814 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebf6f1c-ee94-4002-8560-64c5a0cada01 admin admin] PATH_INFO: `/v3/domains` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:12.245804 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebf6f1c-ee94-4002-8560-64c5a0cada01 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:59:12.246392 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebf6f1c-ee94-4002-8560-64c5a0cada01 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:12.250195 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebf6f1c-ee94-4002-8560-64c5a0cada01 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:12.260412 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d359640a-51e9-42c7-aee9-28035681e2ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:12.263010 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d359640a-51e9-42c7-aee9-28035681e2ba service neutron] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:12.281423 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 827/1657] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 15:59:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 15:59:12.287802 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 831/1658] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Aug 30 15:59:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:59:12.289761 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d359640a-51e9-42c7-aee9-28035681e2ba service neutron] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:12.311657 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d359640a-51e9-42c7-aee9-28035681e2ba service neutron] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:12.346926 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc10858-1e93-4579-bc9b-397a622e135e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:12.347606 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc10858-1e93-4579-bc9b-397a622e135e admin admin] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:12.347890 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc10858-1e93-4579-bc9b-397a622e135e admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:12.348165 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc10858-1e93-4579-bc9b-397a622e135e admin admin] PATH_INFO: `/v3/projects` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:12.348822 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc10858-1e93-4579-bc9b-397a622e135e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:12.354664 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc10858-1e93-4579-bc9b-397a622e135e admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:12.454659 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 828/1659] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Aug 30 15:59:12 2022] POST /identity/v3/projects => generated 377 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 15:59:12.455869 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebf6f1c-ee94-4002-8560-64c5a0cada01 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:12.470267 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebf6f1c-ee94-4002-8560-64c5a0cada01 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:12.675396 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b579e74f-7788-4246-ae2a-cb73d3701da8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:12.676236 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b579e74f-7788-4246-ae2a-cb73d3701da8 admin admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:12.676615 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b579e74f-7788-4246-ae2a-cb73d3701da8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:12.676987 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b579e74f-7788-4246-ae2a-cb73d3701da8 admin admin] PATH_INFO: `/v3/users` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:12.677843 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b579e74f-7788-4246-ae2a-cb73d3701da8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:12.679939 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b579e74f-7788-4246-ae2a-cb73d3701da8 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:12.754572 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 832/1660] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Aug 30 15:59:12 2022] POST /identity/v3/users => generated 289 bytes in 299 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 15:59:12.760130 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc10858-1e93-4579-bc9b-397a622e135e admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:12.772767 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc10858-1e93-4579-bc9b-397a622e135e admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:12.783987 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182a5a2c-49dc-4dcd-9d10-116efcd8c9f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:12.785642 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182a5a2c-49dc-4dcd-9d10-116efcd8c9f8 admin admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:12.785923 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182a5a2c-49dc-4dcd-9d10-116efcd8c9f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:12.786194 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182a5a2c-49dc-4dcd-9d10-116efcd8c9f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:12.786858 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182a5a2c-49dc-4dcd-9d10-116efcd8c9f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:12.791952 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182a5a2c-49dc-4dcd-9d10-116efcd8c9f8 admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:12.817415 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 829/1661] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 30 15:59:12 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 15:59:12.817606 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b579e74f-7788-4246-ae2a-cb73d3701da8 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:12.831235 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b579e74f-7788-4246-ae2a-cb73d3701da8 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:12.846758 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6499a2-5508-4b69-92a0-d89d353144e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:12.847816 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6499a2-5508-4b69-92a0-d89d353144e0 admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:12.847970 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6499a2-5508-4b69-92a0-d89d353144e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:12.848299 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6499a2-5508-4b69-92a0-d89d353144e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:12.849046 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6499a2-5508-4b69-92a0-d89d353144e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:12.850846 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6499a2-5508-4b69-92a0-d89d353144e0 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:12.883119 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 833/1662] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 30 15:59:12 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 15:59:12.885892 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182a5a2c-49dc-4dcd-9d10-116efcd8c9f8 admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:12.900295 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182a5a2c-49dc-4dcd-9d10-116efcd8c9f8 admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:12.910304 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10326eff-a67c-4694-aa60-1e0010dc7afc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:12.914019 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10326eff-a67c-4694-aa60-1e0010dc7afc admin admin] REQUEST_METHOD: `PUT` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:12.914476 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10326eff-a67c-4694-aa60-1e0010dc7afc admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:12.915807 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10326eff-a67c-4694-aa60-1e0010dc7afc admin admin] PATH_INFO: `/v3/projects/f0a12db0c8d94655998f379f0b389312/users/1a8a2a49f97346589cedadae04903627/roles/df8f9f844b1b439181d75a07c1a93418` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:12.956850 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10326eff-a67c-4694-aa60-1e0010dc7afc admin admin] RBAC: Authorizing `identity:create_grant(project_id=f0a12db0c8d94655998f379f0b389312, user_id=1a8a2a49f97346589cedadae04903627, role_id=df8f9f844b1b439181d75a07c1a93418)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:12.964247 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10326eff-a67c-4694-aa60-1e0010dc7afc admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:12.972671 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:59:13.020280 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 830/1663] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 30 15:59:12 2022] PUT /identity/v3/projects/f0a12db0c8d94655998f379f0b389312/users/1a8a2a49f97346589cedadae04903627/roles/df8f9f844b1b439181d75a07c1a93418 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:59:13.022352 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bcf1da-b78e-482e-807f-2fe6e98d125b None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:13.022745 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bcf1da-b78e-482e-807f-2fe6e98d125b None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:13.023119 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bcf1da-b78e-482e-807f-2fe6e98d125b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:13.077487 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-c2bcf1da-b78e-482e-807f-2fe6e98d125b None None] MFA Rules not processed for user `1a8a2a49f97346589cedadae04903627`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:59:13.079802 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-c2bcf1da-b78e-482e-807f-2fe6e98d125b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:59:13.231826 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10326eff-a67c-4694-aa60-1e0010dc7afc admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:13.233946 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-10326eff-a67c-4694-aa60-1e0010dc7afc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:59:13.280961 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 834/1664] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 15:59:13 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:59:13.314166 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bcf1da-b78e-482e-807f-2fe6e98d125b None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:13.431360 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10326eff-a67c-4694-aa60-1e0010dc7afc admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:13.447934 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abfd70f-3a2f-4f65-9f58-e6885662e89e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:13.448920 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abfd70f-3a2f-4f65-9f58-e6885662e89e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:13.449510 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abfd70f-3a2f-4f65-9f58-e6885662e89e admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:13.449898 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abfd70f-3a2f-4f65-9f58-e6885662e89e admin admin] PATH_INFO: `/v3/users/bb2d2ace7bea48938bacf1cf7fbeef5a` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:13.453718 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:59:13.456269 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abfd70f-3a2f-4f65-9f58-e6885662e89e admin admin] RBAC: Authorizing `identity:delete_user(user_id=bb2d2ace7bea48938bacf1cf7fbeef5a)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:13.459097 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abfd70f-3a2f-4f65-9f58-e6885662e89e admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:13.485355 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bcf1da-b78e-482e-807f-2fe6e98d125b None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:13.503814 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e269abb1-cf4e-4352-822e-dd81e552bf3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:13.504942 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e269abb1-cf4e-4352-822e-dd81e552bf3a admin admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:13.505244 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e269abb1-cf4e-4352-822e-dd81e552bf3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:13.505509 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e269abb1-cf4e-4352-822e-dd81e552bf3a admin admin] PATH_INFO: `/v3/projects` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:13.507022 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e269abb1-cf4e-4352-822e-dd81e552bf3a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:13.509161 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e269abb1-cf4e-4352-822e-dd81e552bf3a admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:13.587438 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-6abfd70f-3a2f-4f65-9f58-e6885662e89e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bb2d2ace7bea48938bacf1cf7fbeef5a'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:59:13.609126 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-6abfd70f-3a2f-4f65-9f58-e6885662e89e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bb2d2ace7bea48938bacf1cf7fbeef5a'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:59:13.619111 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 835/1665] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Aug 30 15:59:13 2022] POST /identity/v3/projects => generated 377 bytes in 337 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 15:59:13.623692 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e269abb1-cf4e-4352-822e-dd81e552bf3a admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:13.663178 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-6abfd70f-3a2f-4f65-9f58-e6885662e89e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bb2d2ace7bea48938bacf1cf7fbeef5a'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:59:13.688505 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-6abfd70f-3a2f-4f65-9f58-e6885662e89e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bb2d2ace7bea48938bacf1cf7fbeef5a'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:59:13.689784 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-6abfd70f-3a2f-4f65-9f58-e6885662e89e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bb2d2ace7bea48938bacf1cf7fbeef5a'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:59:13.706708 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 831/1666] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Aug 30 15:59:13 2022] DELETE /identity/v3/users/bb2d2ace7bea48938bacf1cf7fbeef5a => generated 0 bytes in 476 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 15:59:13.708151 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abfd70f-3a2f-4f65-9f58-e6885662e89e admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:13.712856 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-6abfd70f-3a2f-4f65-9f58-e6885662e89e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:59:13.837859 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e269abb1-cf4e-4352-822e-dd81e552bf3a admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:13.943769 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abfd70f-3a2f-4f65-9f58-e6885662e89e admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:13.957837 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8b637d-b56f-4b0c-98c6-04599573f27c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:13.959115 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8b637d-b56f-4b0c-98c6-04599573f27c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:13.959442 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8b637d-b56f-4b0c-98c6-04599573f27c admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:13.959713 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8b637d-b56f-4b0c-98c6-04599573f27c admin admin] PATH_INFO: `/v3/projects/d70595d51b57440f9c3f580281a882fe` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:13.965336 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8b637d-b56f-4b0c-98c6-04599573f27c admin admin] RBAC: Authorizing `identity:delete_project(project_id=d70595d51b57440f9c3f580281a882fe)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:13.969294 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: WARNING py.warnings [None req-9b8b637d-b56f-4b0c-98c6-04599573f27c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 30 15:59:13.969294 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: warnings.warn(msg) Aug 30 15:59:13.969294 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: Aug 30 15:59:13.971858 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8b637d-b56f-4b0c-98c6-04599573f27c admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:14.014289 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbcd394-559c-4ac3-9975-bf3d5074e171 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:14.014927 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbcd394-559c-4ac3-9975-bf3d5074e171 admin admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:14.015245 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbcd394-559c-4ac3-9975-bf3d5074e171 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:14.016856 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbcd394-559c-4ac3-9975-bf3d5074e171 admin admin] PATH_INFO: `/v3/users` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:14.019098 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbcd394-559c-4ac3-9975-bf3d5074e171 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:14.021134 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbcd394-559c-4ac3-9975-bf3d5074e171 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:14.021700 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:INSERT=2,keystone:SELECT=88,keystone:DELETE=10 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:59:14.102766 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 836/1667] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Aug 30 15:59:13 2022] POST /identity/v3/users => generated 295 bytes in 479 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 15:59:14.104281 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8674c031-3ad9-4fed-b5a2-06e19838c7a0 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:14.104561 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8674c031-3ad9-4fed-b5a2-06e19838c7a0 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:14.104826 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8674c031-3ad9-4fed-b5a2-06e19838c7a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:14.127941 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-9b8b637d-b56f-4b0c-98c6-04599573f27c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd70595d51b57440f9c3f580281a882fe', 'request_id': 'req-9b8b637d-b56f-4b0c-98c6-04599573f27c'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:59:14.140328 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-8674c031-3ad9-4fed-b5a2-06e19838c7a0 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:59:14.142970 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-9b8b637d-b56f-4b0c-98c6-04599573f27c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd70595d51b57440f9c3f580281a882fe', 'request_id': 'req-9b8b637d-b56f-4b0c-98c6-04599573f27c'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:59:14.149151 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-8674c031-3ad9-4fed-b5a2-06e19838c7a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:59:14.165817 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-9b8b637d-b56f-4b0c-98c6-04599573f27c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd70595d51b57440f9c3f580281a882fe', 'request_id': 'req-9b8b637d-b56f-4b0c-98c6-04599573f27c'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:59:14.198796 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-9b8b637d-b56f-4b0c-98c6-04599573f27c admin admin] The token cache is being invalidate because project d70595d51b57440f9c3f580281a882fe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81512) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 15:59:14.199826 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-9b8b637d-b56f-4b0c-98c6-04599573f27c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:59:14.200647 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-9b8b637d-b56f-4b0c-98c6-04599573f27c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:59:14.205523 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 832/1668] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Aug 30 15:59:13 2022] DELETE /identity/v3/projects/d70595d51b57440f9c3f580281a882fe => generated 0 bytes in 498 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:59:14.208745 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8b637d-b56f-4b0c-98c6-04599573f27c admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:14.210627 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-9b8b637d-b56f-4b0c-98c6-04599573f27c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:59:14.266348 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 837/1669] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 15:59:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:59:14.270220 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8674c031-3ad9-4fed-b5a2-06e19838c7a0 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:14.282153 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-8674c031-3ad9-4fed-b5a2-06e19838c7a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:59:14.304457 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8674c031-3ad9-4fed-b5a2-06e19838c7a0 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:14.310280 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59733519-853b-4621-bae0-54d6a1e7f4cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:14.310935 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59733519-853b-4621-bae0-54d6a1e7f4cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:14.311215 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59733519-853b-4621-bae0-54d6a1e7f4cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:14.311478 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59733519-853b-4621-bae0-54d6a1e7f4cc admin admin] PATH_INFO: `/v3/users/403edb6da7774ed28abcf65a847d5342` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:14.314213 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59733519-853b-4621-bae0-54d6a1e7f4cc admin admin] RBAC: Authorizing `identity:delete_user(user_id=403edb6da7774ed28abcf65a847d5342)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:14.316351 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59733519-853b-4621-bae0-54d6a1e7f4cc admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:14.330837 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:59:14.405734 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8b637d-b56f-4b0c-98c6-04599573f27c admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:14.419371 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27cb31d-0ba4-45d5-9e10-bb6e5e49d13c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:14.420845 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27cb31d-0ba4-45d5-9e10-bb6e5e49d13c admin admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:14.421120 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27cb31d-0ba4-45d5-9e10-bb6e5e49d13c admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:14.421469 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27cb31d-0ba4-45d5-9e10-bb6e5e49d13c admin admin] PATH_INFO: `/v3/roles` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:14.422059 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27cb31d-0ba4-45d5-9e10-bb6e5e49d13c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:14.427070 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-59733519-853b-4621-bae0-54d6a1e7f4cc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '403edb6da7774ed28abcf65a847d5342'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:59:14.428865 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27cb31d-0ba4-45d5-9e10-bb6e5e49d13c admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:14.467902 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 833/1670] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 30 15:59:14 2022] GET /identity/v3/roles => generated 1420 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 15:59:14.469587 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27cb31d-0ba4-45d5-9e10-bb6e5e49d13c admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:14.469746 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-59733519-853b-4621-bae0-54d6a1e7f4cc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '403edb6da7774ed28abcf65a847d5342'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:59:14.472186 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-f27cb31d-0ba4-45d5-9e10-bb6e5e49d13c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:59:14.503173 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-59733519-853b-4621-bae0-54d6a1e7f4cc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '403edb6da7774ed28abcf65a847d5342'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:59:14.504360 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-59733519-853b-4621-bae0-54d6a1e7f4cc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '403edb6da7774ed28abcf65a847d5342'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:59:14.526444 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-59733519-853b-4621-bae0-54d6a1e7f4cc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '403edb6da7774ed28abcf65a847d5342'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:59:14.545602 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 838/1671] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Aug 30 15:59:14 2022] DELETE /identity/v3/users/403edb6da7774ed28abcf65a847d5342 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:59:14.546252 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59733519-853b-4621-bae0-54d6a1e7f4cc admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:14.550679 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-59733519-853b-4621-bae0-54d6a1e7f4cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:59:14.590477 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27cb31d-0ba4-45d5-9e10-bb6e5e49d13c admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:14.732565 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6d6547-4e3a-444b-857d-c580a0b130df service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f4157105f3488f807b7475008ab722', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:14.734121 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6d6547-4e3a-444b-857d-c580a0b130df service neutron] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:14.734121 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6d6547-4e3a-444b-857d-c580a0b130df service neutron] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:14.734121 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6d6547-4e3a-444b-857d-c580a0b130df service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:14.739366 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-7b6d6547-4e3a-444b-857d-c580a0b130df service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:59:14.777818 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:59:14.840516 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59733519-853b-4621-bae0-54d6a1e7f4cc admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:14.869400 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ef0fd3-ba19-4d36-8f03-92ed324267a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:14.881645 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ef0fd3-ba19-4d36-8f03-92ed324267a0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:14.883222 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ef0fd3-ba19-4d36-8f03-92ed324267a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:14.883582 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ef0fd3-ba19-4d36-8f03-92ed324267a0 admin admin] PATH_INFO: `/v3/projects/db236552acb040b0b97956b3b41f84d0/users/3f04b5b3ace24840b276662f78048dd8/roles/f265386950a641ac8413b53eda4ae34d` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:14.941556 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ef0fd3-ba19-4d36-8f03-92ed324267a0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=db236552acb040b0b97956b3b41f84d0, user_id=3f04b5b3ace24840b276662f78048dd8, role_id=f265386950a641ac8413b53eda4ae34d)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:14.945590 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ef0fd3-ba19-4d36-8f03-92ed324267a0 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:14.982074 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6d6547-4e3a-444b-857d-c580a0b130df service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:14.992245 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6d6547-4e3a-444b-857d-c580a0b130df service neutron] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:15.009683 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 839/1672] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 30 15:59:14 2022] PUT /identity/v3/projects/db236552acb040b0b97956b3b41f84d0/users/3f04b5b3ace24840b276662f78048dd8/roles/f265386950a641ac8413b53eda4ae34d => generated 0 bytes in 464 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:59:15.014382 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ef0fd3-ba19-4d36-8f03-92ed324267a0 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:15.040161 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 834/1673] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 15:59:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 571 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:59:15.162843 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ef0fd3-ba19-4d36-8f03-92ed324267a0 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:15.175302 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476466e7-2b6d-4f86-a86a-3c9ded039447 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:15.176055 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476466e7-2b6d-4f86-a86a-3c9ded039447 admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:15.176218 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476466e7-2b6d-4f86-a86a-3c9ded039447 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:15.176491 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476466e7-2b6d-4f86-a86a-3c9ded039447 admin admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:15.177262 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476466e7-2b6d-4f86-a86a-3c9ded039447 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:15.178992 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476466e7-2b6d-4f86-a86a-3c9ded039447 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:15.208459 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 840/1674] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 30 15:59:15 2022] GET /identity/v3/roles => generated 1420 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 15:59:15.212478 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6d6547-4e3a-444b-857d-c580a0b130df service neutron] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:15.227707 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6d6547-4e3a-444b-857d-c580a0b130df service neutron] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:15.249756 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a16625f-c536-42e0-b892-c117415ec7fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:15.250669 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a16625f-c536-42e0-b892-c117415ec7fe admin admin] REQUEST_METHOD: `PUT` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:15.251064 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a16625f-c536-42e0-b892-c117415ec7fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:15.252619 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a16625f-c536-42e0-b892-c117415ec7fe admin admin] PATH_INFO: `/v3/projects/db236552acb040b0b97956b3b41f84d0/users/3f04b5b3ace24840b276662f78048dd8/roles/f265386950a641ac8413b53eda4ae34d` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:15.257710 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a16625f-c536-42e0-b892-c117415ec7fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=db236552acb040b0b97956b3b41f84d0, user_id=3f04b5b3ace24840b276662f78048dd8, role_id=f265386950a641ac8413b53eda4ae34d)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:15.263348 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a16625f-c536-42e0-b892-c117415ec7fe admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:15.300445 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 835/1675] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 30 15:59:15 2022] PUT /identity/v3/projects/db236552acb040b0b97956b3b41f84d0/users/3f04b5b3ace24840b276662f78048dd8/roles/f265386950a641ac8413b53eda4ae34d => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:59:15.311139 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4ca28d-3443-4be1-b8d2-ac01b9e40cbc None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:15.311433 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4ca28d-3443-4be1-b8d2-ac01b9e40cbc None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:15.311711 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4ca28d-3443-4be1-b8d2-ac01b9e40cbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:15.319293 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:59:15.392754 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-5d4ca28d-3443-4be1-b8d2-ac01b9e40cbc None None] MFA Rules not processed for user `3f04b5b3ace24840b276662f78048dd8`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:59:15.394008 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-5d4ca28d-3443-4be1-b8d2-ac01b9e40cbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:59:15.541585 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 841/1676] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 15:59:15 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 15:59:15.553792 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a16625f-c536-42e0-b892-c117415ec7fe admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:15.556926 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-9a16625f-c536-42e0-b892-c117415ec7fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:59:15.757771 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a16625f-c536-42e0-b892-c117415ec7fe admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:15.783052 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6c3184-fe02-4246-8e63-2554c5190dd5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0100a04821804993866ea20f159663f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:15.789970 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6c3184-fe02-4246-8e63-2554c5190dd5 service cinder] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:15.813563 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6c3184-fe02-4246-8e63-2554c5190dd5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:15.820486 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6c3184-fe02-4246-8e63-2554c5190dd5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:15.832771 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-ea6c3184-fe02-4246-8e63-2554c5190dd5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:59:16.042716 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6c3184-fe02-4246-8e63-2554c5190dd5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:16.045407 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6c3184-fe02-4246-8e63-2554c5190dd5 service cinder] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:16.075804 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 836/1677] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 15:59:15 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 522 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:59:16.950482 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4ca28d-3443-4be1-b8d2-ac01b9e40cbc None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:16.953163 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-5d4ca28d-3443-4be1-b8d2-ac01b9e40cbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:59:17.135922 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4ca28d-3443-4be1-b8d2-ac01b9e40cbc None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:17.148662 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6c3184-fe02-4246-8e63-2554c5190dd5 service cinder] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:17.149180 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e5db73-7124-45f8-adbf-9b226145e537 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:17.149424 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e5db73-7124-45f8-adbf-9b226145e537 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:17.149864 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e5db73-7124-45f8-adbf-9b226145e537 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:17.150295 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e5db73-7124-45f8-adbf-9b226145e537 admin admin] PATH_INFO: `/v3/projects/014b38148dc343d69f56be728fa389b9` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:17.150451 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-ea6c3184-fe02-4246-8e63-2554c5190dd5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:59:17.151735 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e5db73-7124-45f8-adbf-9b226145e537 admin admin] RBAC: Authorizing `identity:delete_project(project_id=014b38148dc343d69f56be728fa389b9)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:17.156062 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e5db73-7124-45f8-adbf-9b226145e537 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:17.173453 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6c3184-fe02-4246-8e63-2554c5190dd5 service cinder] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:17.186810 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d33976-35f6-4c8c-8f92-8e192f5b79dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:17.187476 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d33976-35f6-4c8c-8f92-8e192f5b79dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:17.187753 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d33976-35f6-4c8c-8f92-8e192f5b79dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:17.188024 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d33976-35f6-4c8c-8f92-8e192f5b79dd admin admin] PATH_INFO: `/v3/users/64fe0a3ca5b34e3e9420215a24342974` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:17.194192 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d33976-35f6-4c8c-8f92-8e192f5b79dd admin admin] RBAC: Authorizing `identity:delete_user(user_id=64fe0a3ca5b34e3e9420215a24342974)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:17.196183 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d33976-35f6-4c8c-8f92-8e192f5b79dd admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:17.284806 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-d9e5db73-7124-45f8-adbf-9b226145e537 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '014b38148dc343d69f56be728fa389b9', 'request_id': 'req-d9e5db73-7124-45f8-adbf-9b226145e537'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:59:17.295211 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=93,keystone:INSERT=1,keystone:DELETE=6 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:59:17.304681 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-b4d33976-35f6-4c8c-8f92-8e192f5b79dd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '64fe0a3ca5b34e3e9420215a24342974'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:59:17.317355 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-d9e5db73-7124-45f8-adbf-9b226145e537 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '014b38148dc343d69f56be728fa389b9', 'request_id': 'req-d9e5db73-7124-45f8-adbf-9b226145e537'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:59:17.327446 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-d9e5db73-7124-45f8-adbf-9b226145e537 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '014b38148dc343d69f56be728fa389b9', 'request_id': 'req-d9e5db73-7124-45f8-adbf-9b226145e537'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:59:17.332057 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-b4d33976-35f6-4c8c-8f92-8e192f5b79dd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '64fe0a3ca5b34e3e9420215a24342974'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:59:17.359093 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-d9e5db73-7124-45f8-adbf-9b226145e537 admin admin] The token cache is being invalidate because project 014b38148dc343d69f56be728fa389b9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81511) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 15:59:17.359286 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-d9e5db73-7124-45f8-adbf-9b226145e537 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:59:17.363754 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-d9e5db73-7124-45f8-adbf-9b226145e537 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:59:17.369475 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 842/1678] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Aug 30 15:59:16 2022] DELETE /identity/v3/projects/014b38148dc343d69f56be728fa389b9 => generated 0 bytes in 417 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:59:17.369737 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-b4d33976-35f6-4c8c-8f92-8e192f5b79dd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '64fe0a3ca5b34e3e9420215a24342974'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:59:17.377641 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b676f1-8cc7-4c87-bcbd-6c1597afbefc None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:17.377937 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b676f1-8cc7-4c87-bcbd-6c1597afbefc None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:17.378253 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b676f1-8cc7-4c87-bcbd-6c1597afbefc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:17.393115 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-b4d33976-35f6-4c8c-8f92-8e192f5b79dd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '64fe0a3ca5b34e3e9420215a24342974'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:59:17.395486 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-b4d33976-35f6-4c8c-8f92-8e192f5b79dd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '64fe0a3ca5b34e3e9420215a24342974'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:59:17.421146 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 837/1679] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Aug 30 15:59:17 2022] DELETE /identity/v3/users/64fe0a3ca5b34e3e9420215a24342974 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 15:59:17.421833 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d33976-35f6-4c8c-8f92-8e192f5b79dd admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:17.429643 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-b4d33976-35f6-4c8c-8f92-8e192f5b79dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:59:17.432768 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-d8b676f1-8cc7-4c87-bcbd-6c1597afbefc None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:59:17.434059 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-d8b676f1-8cc7-4c87-bcbd-6c1597afbefc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:59:17.476052 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:59:17.596963 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 843/1680] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 15:59:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 15:59:17.597827 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d33976-35f6-4c8c-8f92-8e192f5b79dd admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:17.599812 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2ce0ac-1535-405a-b5c7-d187c4dc1547 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:17.601552 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2ce0ac-1535-405a-b5c7-d187c4dc1547 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:17.602454 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2ce0ac-1535-405a-b5c7-d187c4dc1547 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:17.616225 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6277d39-c33c-449a-aa53-303c2dc2c01f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:17.616994 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6277d39-c33c-449a-aa53-303c2dc2c01f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:17.617402 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6277d39-c33c-449a-aa53-303c2dc2c01f admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:17.617767 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6277d39-c33c-449a-aa53-303c2dc2c01f admin admin] PATH_INFO: `/v3/projects/ea970bb384dc4e248af61a61fc848945` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:17.622918 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6277d39-c33c-449a-aa53-303c2dc2c01f admin admin] RBAC: Authorizing `identity:delete_project(project_id=ea970bb384dc4e248af61a61fc848945)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:17.632654 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6277d39-c33c-449a-aa53-303c2dc2c01f admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:17.650629 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-1b2ce0ac-1535-405a-b5c7-d187c4dc1547 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:59:17.652112 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-1b2ce0ac-1535-405a-b5c7-d187c4dc1547 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:59:17.666773 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 844/1681] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 15:59:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:59:17.668990 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad04946f-9635-4820-b382-85677abf6178 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:17.669456 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad04946f-9635-4820-b382-85677abf6178 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:17.669888 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad04946f-9635-4820-b382-85677abf6178 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:17.729370 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-ad04946f-9635-4820-b382-85677abf6178 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:59:17.733659 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-ad04946f-9635-4820-b382-85677abf6178 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:59:17.765489 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-b6277d39-c33c-449a-aa53-303c2dc2c01f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ea970bb384dc4e248af61a61fc848945', 'request_id': 'req-b6277d39-c33c-449a-aa53-303c2dc2c01f'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:59:17.802639 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-b6277d39-c33c-449a-aa53-303c2dc2c01f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ea970bb384dc4e248af61a61fc848945', 'request_id': 'req-b6277d39-c33c-449a-aa53-303c2dc2c01f'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:59:17.814661 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-b6277d39-c33c-449a-aa53-303c2dc2c01f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ea970bb384dc4e248af61a61fc848945', 'request_id': 'req-b6277d39-c33c-449a-aa53-303c2dc2c01f'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:59:17.844245 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-b6277d39-c33c-449a-aa53-303c2dc2c01f admin admin] The token cache is being invalidate because project ea970bb384dc4e248af61a61fc848945 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81512) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 15:59:17.844867 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-b6277d39-c33c-449a-aa53-303c2dc2c01f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:59:17.845894 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-b6277d39-c33c-449a-aa53-303c2dc2c01f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:59:17.854332 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 838/1682] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Aug 30 15:59:17 2022] DELETE /identity/v3/projects/ea970bb384dc4e248af61a61fc848945 => generated 0 bytes in 429 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:59:17.857542 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6277d39-c33c-449a-aa53-303c2dc2c01f admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:17.862021 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-b6277d39-c33c-449a-aa53-303c2dc2c01f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:59:17.920737 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 845/1683] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 15:59:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:59:17.922043 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad04946f-9635-4820-b382-85677abf6178 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:17.923604 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-ad04946f-9635-4820-b382-85677abf6178 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:59:17.964460 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad04946f-9635-4820-b382-85677abf6178 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:17.978005 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d94456-a515-490a-9ebc-8459089c635f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:17.982712 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d94456-a515-490a-9ebc-8459089c635f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:17.988066 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d94456-a515-490a-9ebc-8459089c635f admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:17.988626 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d94456-a515-490a-9ebc-8459089c635f admin admin] PATH_INFO: `/v3/users/81ea05967b734878acd2c5b253e12716` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:17.991681 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d94456-a515-490a-9ebc-8459089c635f admin admin] RBAC: Authorizing `identity:delete_user(user_id=81ea05967b734878acd2c5b253e12716)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:18.010985 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d94456-a515-490a-9ebc-8459089c635f admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:18.044246 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:59:18.054710 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6277d39-c33c-449a-aa53-303c2dc2c01f admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:18.068151 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3312c0-7e33-4d31-ab10-35061d0c45eb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f4157105f3488f807b7475008ab722', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:18.068783 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3312c0-7e33-4d31-ab10-35061d0c45eb service neutron] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:18.069055 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3312c0-7e33-4d31-ab10-35061d0c45eb service neutron] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:18.069374 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3312c0-7e33-4d31-ab10-35061d0c45eb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:18.071926 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-6f3312c0-7e33-4d31-ab10-35061d0c45eb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:59:18.139923 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3312c0-7e33-4d31-ab10-35061d0c45eb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:18.149704 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3312c0-7e33-4d31-ab10-35061d0c45eb service neutron] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:18.164688 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-81d94456-a515-490a-9ebc-8459089c635f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '81ea05967b734878acd2c5b253e12716'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:59:18.184850 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-81d94456-a515-490a-9ebc-8459089c635f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '81ea05967b734878acd2c5b253e12716'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:59:18.185901 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 839/1684] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 15:59:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:59:18.189256 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3312c0-7e33-4d31-ab10-35061d0c45eb service neutron] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:18.205098 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-81d94456-a515-490a-9ebc-8459089c635f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '81ea05967b734878acd2c5b253e12716'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:59:18.206697 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-81d94456-a515-490a-9ebc-8459089c635f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '81ea05967b734878acd2c5b253e12716'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:59:18.219845 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3312c0-7e33-4d31-ab10-35061d0c45eb service neutron] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:18.226662 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226c58a7-84d9-4d5d-81c9-a327b35c5076 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:18.227336 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226c58a7-84d9-4d5d-81c9-a327b35c5076 admin admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:18.227692 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226c58a7-84d9-4d5d-81c9-a327b35c5076 admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:18.228064 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226c58a7-84d9-4d5d-81c9-a327b35c5076 admin admin] PATH_INFO: `/v3/domains` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:18.228590 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-81d94456-a515-490a-9ebc-8459089c635f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '81ea05967b734878acd2c5b253e12716'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:59:18.229007 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226c58a7-84d9-4d5d-81c9-a327b35c5076 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81512) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:59:18.230005 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226c58a7-84d9-4d5d-81c9-a327b35c5076 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:18.237603 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226c58a7-84d9-4d5d-81c9-a327b35c5076 admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:18.243644 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 846/1685] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Aug 30 15:59:17 2022] DELETE /identity/v3/users/81ea05967b734878acd2c5b253e12716 => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:59:18.318608 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 840/1686] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Aug 30 15:59:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:59:18.323004 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d94456-a515-490a-9ebc-8459089c635f admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:18.327094 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-81d94456-a515-490a-9ebc-8459089c635f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:59:18.376341 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:59:18.463457 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d94456-a515-490a-9ebc-8459089c635f admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:18.475246 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7920c82a-c72f-491c-9292-e2beb97592c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:18.476268 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7920c82a-c72f-491c-9292-e2beb97592c3 admin admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:18.476743 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7920c82a-c72f-491c-9292-e2beb97592c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:18.477212 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7920c82a-c72f-491c-9292-e2beb97592c3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:18.478225 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7920c82a-c72f-491c-9292-e2beb97592c3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:18.480531 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7920c82a-c72f-491c-9292-e2beb97592c3 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:18.545249 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 847/1687] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Aug 30 15:59:18 2022] POST /identity/v3/projects => generated 389 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 15:59:18.547922 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226c58a7-84d9-4d5d-81c9-a327b35c5076 admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:18.562664 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226c58a7-84d9-4d5d-81c9-a327b35c5076 admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:18.684160 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb0a93e-e3af-442a-a198-5e9e3fcffe47 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:18.684452 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb0a93e-e3af-442a-a198-5e9e3fcffe47 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:18.684722 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb0a93e-e3af-442a-a198-5e9e3fcffe47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:18.687977 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de04034-6ea9-4cad-b115-b45a39310fdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:18.688859 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de04034-6ea9-4cad-b115-b45a39310fdb admin admin] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:18.689379 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de04034-6ea9-4cad-b115-b45a39310fdb admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:18.689953 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de04034-6ea9-4cad-b115-b45a39310fdb admin admin] PATH_INFO: `/v3/users` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:18.690786 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de04034-6ea9-4cad-b115-b45a39310fdb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:18.705439 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de04034-6ea9-4cad-b115-b45a39310fdb admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:18.766821 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 841/1688] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Aug 30 15:59:18 2022] POST /identity/v3/users => generated 295 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 15:59:18.769681 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de04034-6ea9-4cad-b115-b45a39310fdb admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:18.774459 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-8bb0a93e-e3af-442a-a198-5e9e3fcffe47 None None] MFA Rules not processed for user `b133ae684f20468f982ae0c25020d837`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:59:18.775736 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-8bb0a93e-e3af-442a-a198-5e9e3fcffe47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:59:18.790761 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de04034-6ea9-4cad-b115-b45a39310fdb admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:18.811850 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c04dff-cecd-4230-b61e-49cbf24458de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:18.812506 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c04dff-cecd-4230-b61e-49cbf24458de admin admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:18.812771 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c04dff-cecd-4230-b61e-49cbf24458de admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:18.813047 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c04dff-cecd-4230-b61e-49cbf24458de admin admin] PATH_INFO: `/v3/roles` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:18.814763 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c04dff-cecd-4230-b61e-49cbf24458de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:18.816416 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c04dff-cecd-4230-b61e-49cbf24458de admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:18.851225 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 842/1689] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 30 15:59:18 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 15:59:18.852762 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c04dff-cecd-4230-b61e-49cbf24458de admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:18.879779 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c04dff-cecd-4230-b61e-49cbf24458de admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:18.892626 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f852ee8d-995f-45e0-9235-c189e218868b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:18.893301 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f852ee8d-995f-45e0-9235-c189e218868b admin admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:18.893750 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f852ee8d-995f-45e0-9235-c189e218868b admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:18.893867 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f852ee8d-995f-45e0-9235-c189e218868b admin admin] PATH_INFO: `/v3/roles` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:18.894461 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f852ee8d-995f-45e0-9235-c189e218868b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:18.899231 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f852ee8d-995f-45e0-9235-c189e218868b admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:18.927905 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 843/1690] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 30 15:59:18 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 15:59:18.932613 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f852ee8d-995f-45e0-9235-c189e218868b admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:18.973081 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f852ee8d-995f-45e0-9235-c189e218868b admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:18.992257 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 848/1691] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 15:59:18 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 310 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:59:19.003177 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ecd72f-2078-4ebf-b8ee-0f09069902c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:19.003427 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ecd72f-2078-4ebf-b8ee-0f09069902c9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:19.003538 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ecd72f-2078-4ebf-b8ee-0f09069902c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:19.003762 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ecd72f-2078-4ebf-b8ee-0f09069902c9 admin admin] PATH_INFO: `/v3/projects/c613acae655a40a0a7abf63ea3f5990e/users/5bd9533794e7491c8611c1782282b09b/roles/df8f9f844b1b439181d75a07c1a93418` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:19.018491 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb0a93e-e3af-442a-a198-5e9e3fcffe47 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:19.020032 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-8bb0a93e-e3af-442a-a198-5e9e3fcffe47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:59:19.045443 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ecd72f-2078-4ebf-b8ee-0f09069902c9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c613acae655a40a0a7abf63ea3f5990e, user_id=5bd9533794e7491c8611c1782282b09b, role_id=df8f9f844b1b439181d75a07c1a93418)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:19.050884 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ecd72f-2078-4ebf-b8ee-0f09069902c9 admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:19.089507 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 844/1692] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 30 15:59:18 2022] PUT /identity/v3/projects/c613acae655a40a0a7abf63ea3f5990e/users/5bd9533794e7491c8611c1782282b09b/roles/df8f9f844b1b439181d75a07c1a93418 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:59:19.104129 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce293542-676e-4225-93b5-5b26c1a050fe None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:19.105153 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce293542-676e-4225-93b5-5b26c1a050fe None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:19.105842 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce293542-676e-4225-93b5-5b26c1a050fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:19.230809 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb0a93e-e3af-442a-a198-5e9e3fcffe47 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:19.273669 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-ce293542-676e-4225-93b5-5b26c1a050fe None None] MFA Rules not processed for user `5bd9533794e7491c8611c1782282b09b`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:59:19.278446 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-ce293542-676e-4225-93b5-5b26c1a050fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:59:19.421167 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:59:19.532754 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f871d9-0649-4d9a-99b5-0b14f90bb161 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a1e756ae9be4a7bac1e81acf07e8f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:19.533805 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f871d9-0649-4d9a-99b5-0b14f90bb161 service nova] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:19.533805 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f871d9-0649-4d9a-99b5-0b14f90bb161 service nova] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:19.537276 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f871d9-0649-4d9a-99b5-0b14f90bb161 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:19.551476 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 845/1693] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 15:59:19 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 450 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:59:19.552579 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209505c8-38c2-45f5-ba84-55f9d45a1d85 None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:19.552869 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209505c8-38c2-45f5-ba84-55f9d45a1d85 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:19.553139 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209505c8-38c2-45f5-ba84-55f9d45a1d85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:19.609338 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:59:19.634763 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-209505c8-38c2-45f5-ba84-55f9d45a1d85 None None] MFA Rules not processed for user `096f60ad096243d3829d256db17a5f8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:59:19.642440 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-209505c8-38c2-45f5-ba84-55f9d45a1d85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:59:19.676698 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f871d9-0649-4d9a-99b5-0b14f90bb161 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:19.678675 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f871d9-0649-4d9a-99b5-0b14f90bb161 service nova] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:19.714702 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 849/1694] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 15:59:19 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 697 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 15:59:19.715783 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f871d9-0649-4d9a-99b5-0b14f90bb161 service nova] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:19.833813 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 846/1695] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 15:59:19 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 282 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:59:19.850992 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b266083-0c47-4919-bb4a-91cb27b20c5b None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:19.851305 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b266083-0c47-4919-bb4a-91cb27b20c5b None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:19.851550 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b266083-0c47-4919-bb4a-91cb27b20c5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:19.886995 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f871d9-0649-4d9a-99b5-0b14f90bb161 service nova] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:19.893260 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c941fba2-6710-46e0-8557-ef3b24a52f97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:19.895282 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c941fba2-6710-46e0-8557-ef3b24a52f97 admin admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:19.895733 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c941fba2-6710-46e0-8557-ef3b24a52f97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:19.896773 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c941fba2-6710-46e0-8557-ef3b24a52f97 admin admin] PATH_INFO: `/v3/projects` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:19.897720 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c941fba2-6710-46e0-8557-ef3b24a52f97 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:19.900965 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c941fba2-6710-46e0-8557-ef3b24a52f97 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:19.932157 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-0b266083-0c47-4919-bb4a-91cb27b20c5b None None] MFA Rules not processed for user `096f60ad096243d3829d256db17a5f8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:59:19.945250 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-0b266083-0c47-4919-bb4a-91cb27b20c5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:59:19.970176 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 850/1696] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Aug 30 15:59:19 2022] POST /identity/v3/projects => generated 389 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 15:59:19.974019 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c941fba2-6710-46e0-8557-ef3b24a52f97 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:20.015406 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 847/1697] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 15:59:19 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:59:20.033268 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b266083-0c47-4919-bb4a-91cb27b20c5b None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:20.035468 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-0b266083-0c47-4919-bb4a-91cb27b20c5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:59:20.088377 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b266083-0c47-4919-bb4a-91cb27b20c5b None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:20.213500 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c941fba2-6710-46e0-8557-ef3b24a52f97 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:20.219306 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d6a5a1-e0a9-4a83-ba33-7fc5a54ddb33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:20.219933 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d6a5a1-e0a9-4a83-ba33-7fc5a54ddb33 admin admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:20.220215 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d6a5a1-e0a9-4a83-ba33-7fc5a54ddb33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:20.220487 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d6a5a1-e0a9-4a83-ba33-7fc5a54ddb33 admin admin] PATH_INFO: `/v3/users` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:20.221326 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d6a5a1-e0a9-4a83-ba33-7fc5a54ddb33 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:20.230181 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d6a5a1-e0a9-4a83-ba33-7fc5a54ddb33 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:20.243393 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc57b68a-540c-40a6-a09e-1ca517387166 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a1e756ae9be4a7bac1e81acf07e8f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:20.244030 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc57b68a-540c-40a6-a09e-1ca517387166 service nova] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:20.244299 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc57b68a-540c-40a6-a09e-1ca517387166 service nova] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:20.246243 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc57b68a-540c-40a6-a09e-1ca517387166 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:20.247791 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:59:20.285186 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 851/1698] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Aug 30 15:59:19 2022] POST /identity/v3/users => generated 301 bytes in 313 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 15:59:20.286267 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d6a5a1-e0a9-4a83-ba33-7fc5a54ddb33 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:20.315024 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d6a5a1-e0a9-4a83-ba33-7fc5a54ddb33 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:20.330049 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5f1d97-b5a6-47fb-a76c-d10a2c679145 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:20.330698 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5f1d97-b5a6-47fb-a76c-d10a2c679145 admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:20.330986 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5f1d97-b5a6-47fb-a76c-d10a2c679145 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:20.331258 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5f1d97-b5a6-47fb-a76c-d10a2c679145 admin admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:20.331919 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5f1d97-b5a6-47fb-a76c-d10a2c679145 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:20.333606 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5f1d97-b5a6-47fb-a76c-d10a2c679145 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:20.364700 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 852/1699] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 30 15:59:20 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 15:59:20.373733 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5f1d97-b5a6-47fb-a76c-d10a2c679145 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:20.379015 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-7f5f1d97-b5a6-47fb-a76c-d10a2c679145 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:59:20.409031 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5f1d97-b5a6-47fb-a76c-d10a2c679145 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:20.421908 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3980a396-e8f2-4b7c-9532-3ae6d4c4f4f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:20.422722 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3980a396-e8f2-4b7c-9532-3ae6d4c4f4f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:20.422894 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3980a396-e8f2-4b7c-9532-3ae6d4c4f4f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:20.423169 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3980a396-e8f2-4b7c-9532-3ae6d4c4f4f6 admin admin] PATH_INFO: `/v3/projects/2b98d7e00362424f9f7d9044fd391a3c` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:20.424557 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3980a396-e8f2-4b7c-9532-3ae6d4c4f4f6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2b98d7e00362424f9f7d9044fd391a3c)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:20.426641 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3980a396-e8f2-4b7c-9532-3ae6d4c4f4f6 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:20.491975 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc57b68a-540c-40a6-a09e-1ca517387166 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:20.494264 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc57b68a-540c-40a6-a09e-1ca517387166 service nova] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:20.531714 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 848/1700] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 15:59:20 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 499 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:59:20.533615 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc57b68a-540c-40a6-a09e-1ca517387166 service nova] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:20.538430 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-bc57b68a-540c-40a6-a09e-1ca517387166 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:59:20.575292 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc57b68a-540c-40a6-a09e-1ca517387166 service nova] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:20.597377 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-3980a396-e8f2-4b7c-9532-3ae6d4c4f4f6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2b98d7e00362424f9f7d9044fd391a3c', 'request_id': 'req-3980a396-e8f2-4b7c-9532-3ae6d4c4f4f6'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:59:20.623759 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-3980a396-e8f2-4b7c-9532-3ae6d4c4f4f6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2b98d7e00362424f9f7d9044fd391a3c', 'request_id': 'req-3980a396-e8f2-4b7c-9532-3ae6d4c4f4f6'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:59:20.647170 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-3980a396-e8f2-4b7c-9532-3ae6d4c4f4f6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2b98d7e00362424f9f7d9044fd391a3c', 'request_id': 'req-3980a396-e8f2-4b7c-9532-3ae6d4c4f4f6'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:59:20.662321 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:59:20.670847 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-3980a396-e8f2-4b7c-9532-3ae6d4c4f4f6 admin admin] The token cache is being invalidate because project 2b98d7e00362424f9f7d9044fd391a3c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81511) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 15:59:20.670847 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-3980a396-e8f2-4b7c-9532-3ae6d4c4f4f6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:59:20.671156 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-3980a396-e8f2-4b7c-9532-3ae6d4c4f4f6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:59:20.675873 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 853/1701] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Aug 30 15:59:20 2022] DELETE /identity/v3/projects/2b98d7e00362424f9f7d9044fd391a3c => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:59:20.677147 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3980a396-e8f2-4b7c-9532-3ae6d4c4f4f6 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:20.680203 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-3980a396-e8f2-4b7c-9532-3ae6d4c4f4f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:59:20.743579 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ddaa02-722f-485b-8d69-672ab1f6d0c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:20.744520 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ddaa02-722f-485b-8d69-672ab1f6d0c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:20.745233 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ddaa02-722f-485b-8d69-672ab1f6d0c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:20.745651 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ddaa02-722f-485b-8d69-672ab1f6d0c5 admin admin] PATH_INFO: `/v3/projects/08146850cb014c16ada76e8252f9b149` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:20.748060 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ddaa02-722f-485b-8d69-672ab1f6d0c5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=08146850cb014c16ada76e8252f9b149)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:20.757077 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ddaa02-722f-485b-8d69-672ab1f6d0c5 admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:20.855035 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3980a396-e8f2-4b7c-9532-3ae6d4c4f4f6 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:20.863569 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc3707d-e20e-4df9-b0e5-849de8342fc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:20.864657 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc3707d-e20e-4df9-b0e5-849de8342fc8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:20.864933 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc3707d-e20e-4df9-b0e5-849de8342fc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:20.865201 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc3707d-e20e-4df9-b0e5-849de8342fc8 admin admin] PATH_INFO: `/v3/projects/2e782d04ce9a493d892471610b95c474/users/fa6c589200b840979a3ea558331786f6/roles/f265386950a641ac8413b53eda4ae34d` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:20.885435 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc3707d-e20e-4df9-b0e5-849de8342fc8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e782d04ce9a493d892471610b95c474, user_id=fa6c589200b840979a3ea558331786f6, role_id=f265386950a641ac8413b53eda4ae34d)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:20.888494 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc3707d-e20e-4df9-b0e5-849de8342fc8 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:20.912102 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-53ddaa02-722f-485b-8d69-672ab1f6d0c5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '08146850cb014c16ada76e8252f9b149', 'request_id': 'req-53ddaa02-722f-485b-8d69-672ab1f6d0c5'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:59:20.919509 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 854/1702] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 30 15:59:20 2022] PUT /identity/v3/projects/2e782d04ce9a493d892471610b95c474/users/fa6c589200b840979a3ea558331786f6/roles/f265386950a641ac8413b53eda4ae34d => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:59:20.922880 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98abc8e8-d534-4cd8-a120-bddc8b88a88a None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:20.923166 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98abc8e8-d534-4cd8-a120-bddc8b88a88a None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:20.924564 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98abc8e8-d534-4cd8-a120-bddc8b88a88a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:20.936321 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-53ddaa02-722f-485b-8d69-672ab1f6d0c5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '08146850cb014c16ada76e8252f9b149', 'request_id': 'req-53ddaa02-722f-485b-8d69-672ab1f6d0c5'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:59:20.947558 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-53ddaa02-722f-485b-8d69-672ab1f6d0c5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '08146850cb014c16ada76e8252f9b149', 'request_id': 'req-53ddaa02-722f-485b-8d69-672ab1f6d0c5'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:59:20.968257 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-53ddaa02-722f-485b-8d69-672ab1f6d0c5 admin admin] The token cache is being invalidate because project 08146850cb014c16ada76e8252f9b149 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81512) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 15:59:20.968556 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-53ddaa02-722f-485b-8d69-672ab1f6d0c5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:59:20.972167 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-53ddaa02-722f-485b-8d69-672ab1f6d0c5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:59:20.974392 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-98abc8e8-d534-4cd8-a120-bddc8b88a88a None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:59:20.975645 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-98abc8e8-d534-4cd8-a120-bddc8b88a88a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:59:20.976778 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 849/1703] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Aug 30 15:59:20 2022] DELETE /identity/v3/projects/08146850cb014c16ada76e8252f9b149 => generated 0 bytes in 443 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:59:20.977788 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ddaa02-722f-485b-8d69-672ab1f6d0c5 admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:20.979110 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-53ddaa02-722f-485b-8d69-672ab1f6d0c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:59:21.050444 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:INSERT=5,keystone:SELECT=88,keystone:DELETE=7 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:59:21.099434 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 855/1704] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 15:59:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:59:21.100803 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd9119b-f7e7-4a1d-9058-f893d6a34c14 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:21.101088 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd9119b-f7e7-4a1d-9058-f893d6a34c14 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:21.105853 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd9119b-f7e7-4a1d-9058-f893d6a34c14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:21.145976 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-4cd9119b-f7e7-4a1d-9058-f893d6a34c14 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:59:21.154376 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-4cd9119b-f7e7-4a1d-9058-f893d6a34c14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:59:21.175950 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 856/1705] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 15:59:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:59:21.177391 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc31cfe-4a42-412d-a8be-3d2951cc7a59 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:21.177665 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc31cfe-4a42-412d-a8be-3d2951cc7a59 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:21.177939 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc31cfe-4a42-412d-a8be-3d2951cc7a59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:21.200350 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ddaa02-722f-485b-8d69-672ab1f6d0c5 admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:21.219469 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f577373-4cca-43d9-b166-d7b8b58f62cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:21.221687 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f577373-4cca-43d9-b166-d7b8b58f62cb admin admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:21.222042 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f577373-4cca-43d9-b166-d7b8b58f62cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:21.223601 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-efc31cfe-4a42-412d-a8be-3d2951cc7a59 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:59:21.225168 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f577373-4cca-43d9-b166-d7b8b58f62cb admin admin] PATH_INFO: `/v3/roles` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:21.226501 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f577373-4cca-43d9-b166-d7b8b58f62cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:21.231619 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-efc31cfe-4a42-412d-a8be-3d2951cc7a59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:59:21.248169 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f577373-4cca-43d9-b166-d7b8b58f62cb admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:21.267599 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 857/1706] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 15:59:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:59:21.278054 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b29ce64-ed1c-405d-b7ba-c5fb4bfec78a None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:21.278386 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b29ce64-ed1c-405d-b7ba-c5fb4bfec78a None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:21.278663 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b29ce64-ed1c-405d-b7ba-c5fb4bfec78a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:21.302867 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 850/1707] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 30 15:59:20 2022] GET /identity/v3/roles => generated 1420 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 15:59:21.304198 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f577373-4cca-43d9-b166-d7b8b58f62cb admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:21.339165 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f577373-4cca-43d9-b166-d7b8b58f62cb admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:21.357081 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-7b29ce64-ed1c-405d-b7ba-c5fb4bfec78a None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:59:21.359638 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-7b29ce64-ed1c-405d-b7ba-c5fb4bfec78a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:59:21.363371 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b665973-cdac-4ab6-b300-075bcbf5999e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:21.365529 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b665973-cdac-4ab6-b300-075bcbf5999e admin admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:21.366015 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b665973-cdac-4ab6-b300-075bcbf5999e admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:21.366663 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b665973-cdac-4ab6-b300-075bcbf5999e admin admin] PATH_INFO: `/v3/domains` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:21.367678 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b665973-cdac-4ab6-b300-075bcbf5999e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81512) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:59:21.368177 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b665973-cdac-4ab6-b300-075bcbf5999e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:21.371750 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b665973-cdac-4ab6-b300-075bcbf5999e admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:21.396642 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 858/1708] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 15:59:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:59:21.397122 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 851/1709] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Aug 30 15:59:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:59:21.397552 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b29ce64-ed1c-405d-b7ba-c5fb4bfec78a None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:21.401930 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b665973-cdac-4ab6-b300-075bcbf5999e admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:21.437114 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b29ce64-ed1c-405d-b7ba-c5fb4bfec78a None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:21.441601 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b665973-cdac-4ab6-b300-075bcbf5999e admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:21.453265 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701c8a23-cfde-4c5d-be8e-550fe2892b54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:21.456817 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701c8a23-cfde-4c5d-be8e-550fe2892b54 admin admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:21.457442 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701c8a23-cfde-4c5d-be8e-550fe2892b54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:21.457866 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701c8a23-cfde-4c5d-be8e-550fe2892b54 admin admin] PATH_INFO: `/v3/domains` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:21.462714 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4a29da-4877-489c-a0b5-baf173f91e23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:21.463404 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4a29da-4877-489c-a0b5-baf173f91e23 admin admin] REQUEST_METHOD: `PUT` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:21.463675 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4a29da-4877-489c-a0b5-baf173f91e23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:21.463952 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4a29da-4877-489c-a0b5-baf173f91e23 admin admin] PATH_INFO: `/v3/projects/2e782d04ce9a493d892471610b95c474/users/fa6c589200b840979a3ea558331786f6/roles/f265386950a641ac8413b53eda4ae34d` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:21.466265 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701c8a23-cfde-4c5d-be8e-550fe2892b54 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81512) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:59:21.467116 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4a29da-4877-489c-a0b5-baf173f91e23 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e782d04ce9a493d892471610b95c474, user_id=fa6c589200b840979a3ea558331786f6, role_id=f265386950a641ac8413b53eda4ae34d)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:21.469424 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701c8a23-cfde-4c5d-be8e-550fe2892b54 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:21.470091 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4a29da-4877-489c-a0b5-baf173f91e23 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:21.476462 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701c8a23-cfde-4c5d-be8e-550fe2892b54 admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:21.521573 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 859/1710] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 30 15:59:21 2022] PUT /identity/v3/projects/2e782d04ce9a493d892471610b95c474/users/fa6c589200b840979a3ea558331786f6/roles/f265386950a641ac8413b53eda4ae34d => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:59:21.521573 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4a29da-4877-489c-a0b5-baf173f91e23 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:21.531385 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 852/1711] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Aug 30 15:59:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:59:21.531385 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9f8d0a-d28b-4f03-bb28-9e80b7e3be3f None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:21.531385 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9f8d0a-d28b-4f03-bb28-9e80b7e3be3f None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:21.531385 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9f8d0a-d28b-4f03-bb28-9e80b7e3be3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:21.621480 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-6f9f8d0a-d28b-4f03-bb28-9e80b7e3be3f None None] MFA Rules not processed for user `fa6c589200b840979a3ea558331786f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:59:21.623758 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-6f9f8d0a-d28b-4f03-bb28-9e80b7e3be3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:59:21.711715 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:59:21.720825 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4a29da-4877-489c-a0b5-baf173f91e23 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:21.727957 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c40e05-84e7-4d05-98dd-ec3fe9ffa756 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:21.728867 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c40e05-84e7-4d05-98dd-ec3fe9ffa756 admin admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:21.729325 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c40e05-84e7-4d05-98dd-ec3fe9ffa756 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:21.729749 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c40e05-84e7-4d05-98dd-ec3fe9ffa756 admin admin] PATH_INFO: `/v3/projects` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:21.730566 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c40e05-84e7-4d05-98dd-ec3fe9ffa756 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:21.743282 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c40e05-84e7-4d05-98dd-ec3fe9ffa756 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:21.826681 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 860/1712] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Aug 30 15:59:21 2022] POST /identity/v3/projects => generated 375 bytes in 311 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 15:59:21.827478 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c40e05-84e7-4d05-98dd-ec3fe9ffa756 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:21.859711 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 853/1713] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 15:59:21 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 331 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:59:21.862373 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9f8d0a-d28b-4f03-bb28-9e80b7e3be3f None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:21.877018 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-6f9f8d0a-d28b-4f03-bb28-9e80b7e3be3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:59:21.976599 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c40e05-84e7-4d05-98dd-ec3fe9ffa756 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:21.994352 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20ba84b-9425-4d7e-8e35-538dc7d1dca5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:21.995279 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20ba84b-9425-4d7e-8e35-538dc7d1dca5 admin admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:21.996405 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20ba84b-9425-4d7e-8e35-538dc7d1dca5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:21.996842 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20ba84b-9425-4d7e-8e35-538dc7d1dca5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:21.999084 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20ba84b-9425-4d7e-8e35-538dc7d1dca5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:22.001758 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20ba84b-9425-4d7e-8e35-538dc7d1dca5 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:22.007986 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9f8d0a-d28b-4f03-bb28-9e80b7e3be3f None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:22.073633 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 861/1714] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Aug 30 15:59:21 2022] POST /identity/v3/projects => generated 405 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 15:59:22.076316 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20ba84b-9425-4d7e-8e35-538dc7d1dca5 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:22.131499 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:59:22.132403 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1320b8a1-663f-42b0-8bc0-ab0e021a7005 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0100a04821804993866ea20f159663f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:22.132783 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1320b8a1-663f-42b0-8bc0-ab0e021a7005 service cinder] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:22.133066 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1320b8a1-663f-42b0-8bc0-ab0e021a7005 service cinder] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:22.133364 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1320b8a1-663f-42b0-8bc0-ab0e021a7005 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:22.139313 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-1320b8a1-663f-42b0-8bc0-ab0e021a7005 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:59:22.224335 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20ba84b-9425-4d7e-8e35-538dc7d1dca5 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:22.235819 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c44160f-d94b-46e5-b1e8-98fd419e3501 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:22.236631 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c44160f-d94b-46e5-b1e8-98fd419e3501 admin admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:22.236734 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c44160f-d94b-46e5-b1e8-98fd419e3501 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:22.236968 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c44160f-d94b-46e5-b1e8-98fd419e3501 admin admin] PATH_INFO: `/v3/users` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:22.237671 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c44160f-d94b-46e5-b1e8-98fd419e3501 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:22.242928 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c44160f-d94b-46e5-b1e8-98fd419e3501 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:22.302354 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 862/1715] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Aug 30 15:59:22 2022] POST /identity/v3/users => generated 288 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 15:59:22.303574 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c44160f-d94b-46e5-b1e8-98fd419e3501 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:22.306694 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1320b8a1-663f-42b0-8bc0-ab0e021a7005 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:22.309470 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-5c44160f-d94b-46e5-b1e8-98fd419e3501 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:59:22.311015 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1320b8a1-663f-42b0-8bc0-ab0e021a7005 service cinder] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:22.348773 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 854/1716] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 15:59:21 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 489 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:59:22.349978 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ab678e-2a49-4841-bbb3-f0cd3094711a None None] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:22.350269 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ab678e-2a49-4841-bbb3-f0cd3094711a None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:22.350629 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ab678e-2a49-4841-bbb3-f0cd3094711a None None] PATH_INFO: `/` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:22.362670 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 855/1717] 10.222.0.20 () {54 vars in 912 bytes} [Tue Aug 30 15:59:22 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 15:59:22.363615 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ab678e-2a49-4841-bbb3-f0cd3094711a None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:22.445193 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c44160f-d94b-46e5-b1e8-98fd419e3501 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:22.461914 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ab678e-2a49-4841-bbb3-f0cd3094711a None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:22.467486 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf646ef6-1696-46a0-97c6-575140f5a3a6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0100a04821804993866ea20f159663f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:22.468192 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf646ef6-1696-46a0-97c6-575140f5a3a6 service cinder] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:22.468905 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf646ef6-1696-46a0-97c6-575140f5a3a6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:22.468905 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf646ef6-1696-46a0-97c6-575140f5a3a6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:22.471544 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9ec8f1-f668-4158-8f39-9de18c75dfa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:22.473122 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9ec8f1-f668-4158-8f39-9de18c75dfa9 admin admin] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:22.473689 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9ec8f1-f668-4158-8f39-9de18c75dfa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:22.474150 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9ec8f1-f668-4158-8f39-9de18c75dfa9 admin admin] PATH_INFO: `/v3/users` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:22.477248 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9ec8f1-f668-4158-8f39-9de18c75dfa9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:22.480263 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9ec8f1-f668-4158-8f39-9de18c75dfa9 admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:22.548021 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 856/1718] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Aug 30 15:59:22 2022] POST /identity/v3/users => generated 303 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 15:59:22.552753 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9ec8f1-f668-4158-8f39-9de18c75dfa9 admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:22.571812 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:59:22.580553 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9ec8f1-f668-4158-8f39-9de18c75dfa9 admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:22.592915 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc8bc3e-d876-4bcf-a489-6e7cd5cf39cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:22.594283 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc8bc3e-d876-4bcf-a489-6e7cd5cf39cf admin admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:22.595441 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc8bc3e-d876-4bcf-a489-6e7cd5cf39cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:22.595734 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc8bc3e-d876-4bcf-a489-6e7cd5cf39cf admin admin] PATH_INFO: `/v3/roles` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:22.596385 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc8bc3e-d876-4bcf-a489-6e7cd5cf39cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:22.599669 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc8bc3e-d876-4bcf-a489-6e7cd5cf39cf admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:22.648494 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 857/1719] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 30 15:59:22 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 15:59:22.658410 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e4384c-b818-49b4-9a46-ec8b4dd9abb0 None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:22.658410 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e4384c-b818-49b4-9a46-ec8b4dd9abb0 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:22.658410 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e4384c-b818-49b4-9a46-ec8b4dd9abb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:22.696887 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf646ef6-1696-46a0-97c6-575140f5a3a6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:22.700351 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf646ef6-1696-46a0-97c6-575140f5a3a6 service cinder] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:22.730947 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 863/1720] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 15:59:22 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:59:22.732066 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf646ef6-1696-46a0-97c6-575140f5a3a6 service cinder] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:22.771948 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf646ef6-1696-46a0-97c6-575140f5a3a6 service cinder] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:22.773544 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-31e4384c-b818-49b4-9a46-ec8b4dd9abb0 None None] MFA Rules not processed for user `65f4157105f3488f807b7475008ab722`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:59:22.775992 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-31e4384c-b818-49b4-9a46-ec8b4dd9abb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:59:22.789279 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91499f1d-9134-4fc8-9052-a128b3d30de1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:22.789926 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91499f1d-9134-4fc8-9052-a128b3d30de1 admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:22.790208 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91499f1d-9134-4fc8-9052-a128b3d30de1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:22.790521 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91499f1d-9134-4fc8-9052-a128b3d30de1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:22.791185 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91499f1d-9134-4fc8-9052-a128b3d30de1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:22.792804 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91499f1d-9134-4fc8-9052-a128b3d30de1 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:22.900940 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 864/1721] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 30 15:59:22 2022] GET /identity/v3/roles => generated 1420 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 15:59:22.902633 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f3d3d4-980d-46a7-8270-a6e2e406ec23 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:22.902922 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f3d3d4-980d-46a7-8270-a6e2e406ec23 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:22.903195 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f3d3d4-980d-46a7-8270-a6e2e406ec23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:22.956371 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-d5f3d3d4-980d-46a7-8270-a6e2e406ec23 None None] MFA Rules not processed for user `252dd6215082420cade91b100c73f100`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:59:22.959511 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-d5f3d3d4-980d-46a7-8270-a6e2e406ec23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:59:22.978320 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 858/1722] 10.222.0.20 () {58 vars in 1009 bytes} [Tue Aug 30 15:59:22 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 329 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:59:22.979157 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e4384c-b818-49b4-9a46-ec8b4dd9abb0 None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:23.014371 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e4384c-b818-49b4-9a46-ec8b4dd9abb0 None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:23.022206 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e255072-aada-4bab-bdd8-4e3254add365 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:23.022846 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e255072-aada-4bab-bdd8-4e3254add365 admin admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:23.023126 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e255072-aada-4bab-bdd8-4e3254add365 admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:23.023414 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e255072-aada-4bab-bdd8-4e3254add365 admin admin] PATH_INFO: `/v3/roles` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:23.024065 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e255072-aada-4bab-bdd8-4e3254add365 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:23.025777 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e255072-aada-4bab-bdd8-4e3254add365 admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:23.079523 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 859/1723] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 30 15:59:22 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 15:59:23.080506 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e255072-aada-4bab-bdd8-4e3254add365 admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:23.136125 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e255072-aada-4bab-bdd8-4e3254add365 admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:23.162280 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7feca578-e308-4050-8a7a-64bad96e32ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:23.162637 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7feca578-e308-4050-8a7a-64bad96e32ed admin admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:23.162908 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7feca578-e308-4050-8a7a-64bad96e32ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:23.163169 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7feca578-e308-4050-8a7a-64bad96e32ed admin admin] PATH_INFO: `/v3/roles` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:23.165638 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7feca578-e308-4050-8a7a-64bad96e32ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:23.170211 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7feca578-e308-4050-8a7a-64bad96e32ed admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:23.171233 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:59:23.193556 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 860/1724] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 30 15:59:23 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 15:59:23.195621 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9033f4e7-1171-48c4-b0c0-57db6d1edffd None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:23.198462 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9033f4e7-1171-48c4-b0c0-57db6d1edffd None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:23.201566 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9033f4e7-1171-48c4-b0c0-57db6d1edffd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:23.234100 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 865/1725] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 15:59:22 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 332 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:59:23.235663 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f3d3d4-980d-46a7-8270-a6e2e406ec23 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:23.259936 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f3d3d4-980d-46a7-8270-a6e2e406ec23 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:23.270816 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-9033f4e7-1171-48c4-b0c0-57db6d1edffd None None] MFA Rules not processed for user `c34836a75f144f74aa3fd6a38ccb9791`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:59:23.274504 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-9033f4e7-1171-48c4-b0c0-57db6d1edffd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:59:23.281868 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2eea148-533c-4882-b40e-c785ce628a74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:23.281868 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2eea148-533c-4882-b40e-c785ce628a74 admin admin] REQUEST_METHOD: `PUT` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:23.281868 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2eea148-533c-4882-b40e-c785ce628a74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:23.281868 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2eea148-533c-4882-b40e-c785ce628a74 admin admin] PATH_INFO: `/v3/projects/febac92c68d64347bf169aecd6633f7d/users/1a8a5ef37e0e4bb48b40e542aa376b3a/roles/df8f9f844b1b439181d75a07c1a93418` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:23.325864 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2eea148-533c-4882-b40e-c785ce628a74 admin admin] RBAC: Authorizing `identity:create_grant(project_id=febac92c68d64347bf169aecd6633f7d, user_id=1a8a5ef37e0e4bb48b40e542aa376b3a, role_id=df8f9f844b1b439181d75a07c1a93418)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:23.340005 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2eea148-533c-4882-b40e-c785ce628a74 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:23.367193 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 866/1726] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 30 15:59:23 2022] PUT /identity/v3/projects/febac92c68d64347bf169aecd6633f7d/users/1a8a5ef37e0e4bb48b40e542aa376b3a/roles/df8f9f844b1b439181d75a07c1a93418 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:59:23.368611 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2eea148-533c-4882-b40e-c785ce628a74 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:23.399390 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 861/1727] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 15:59:23 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:59:23.400330 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9033f4e7-1171-48c4-b0c0-57db6d1edffd None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:23.422078 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9033f4e7-1171-48c4-b0c0-57db6d1edffd None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:23.462325 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2eea148-533c-4882-b40e-c785ce628a74 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:23.477860 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588de6b9-e438-4c49-b6e5-115cb88e02b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:23.479154 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588de6b9-e438-4c49-b6e5-115cb88e02b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:23.479479 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588de6b9-e438-4c49-b6e5-115cb88e02b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:23.479880 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588de6b9-e438-4c49-b6e5-115cb88e02b9 admin admin] PATH_INFO: `/v3/users/5bd9533794e7491c8611c1782282b09b` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:23.481806 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588de6b9-e438-4c49-b6e5-115cb88e02b9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5bd9533794e7491c8611c1782282b09b)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:23.483598 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588de6b9-e438-4c49-b6e5-115cb88e02b9 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:23.501519 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:59:23.513930 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf29179f-9ec5-4570-805e-d50132026a0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:23.514702 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf29179f-9ec5-4570-805e-d50132026a0b admin admin] REQUEST_METHOD: `PUT` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:23.515653 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf29179f-9ec5-4570-805e-d50132026a0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:23.515928 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf29179f-9ec5-4570-805e-d50132026a0b admin admin] PATH_INFO: `/v3/projects/7c37c789ed174f33800e857f92cc69a8/users/3b9a1e0efb444d40bcce2e9db2c4d347/roles/df8f9f844b1b439181d75a07c1a93418` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:23.540343 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf29179f-9ec5-4570-805e-d50132026a0b admin admin] RBAC: Authorizing `identity:create_grant(project_id=7c37c789ed174f33800e857f92cc69a8, user_id=3b9a1e0efb444d40bcce2e9db2c4d347, role_id=df8f9f844b1b439181d75a07c1a93418)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:23.543844 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf29179f-9ec5-4570-805e-d50132026a0b admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:23.574102 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 862/1728] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 30 15:59:23 2022] PUT /identity/v3/projects/7c37c789ed174f33800e857f92cc69a8/users/3b9a1e0efb444d40bcce2e9db2c4d347/roles/df8f9f844b1b439181d75a07c1a93418 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:59:23.575541 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf29179f-9ec5-4570-805e-d50132026a0b admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:23.577355 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-cf29179f-9ec5-4570-805e-d50132026a0b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:59:23.586766 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-588de6b9-e438-4c49-b6e5-115cb88e02b9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5bd9533794e7491c8611c1782282b09b'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:59:23.608871 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-588de6b9-e438-4c49-b6e5-115cb88e02b9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5bd9533794e7491c8611c1782282b09b'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:59:23.634394 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-588de6b9-e438-4c49-b6e5-115cb88e02b9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5bd9533794e7491c8611c1782282b09b'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:59:23.635160 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-588de6b9-e438-4c49-b6e5-115cb88e02b9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5bd9533794e7491c8611c1782282b09b'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:59:23.645409 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-588de6b9-e438-4c49-b6e5-115cb88e02b9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5bd9533794e7491c8611c1782282b09b'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:59:23.657789 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 867/1729] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Aug 30 15:59:23 2022] DELETE /identity/v3/users/5bd9533794e7491c8611c1782282b09b => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:59:23.661168 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588de6b9-e438-4c49-b6e5-115cb88e02b9 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:23.665587 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-588de6b9-e438-4c49-b6e5-115cb88e02b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:59:23.695502 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf29179f-9ec5-4570-805e-d50132026a0b admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:23.748054 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:59:23.784275 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588de6b9-e438-4c49-b6e5-115cb88e02b9 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:23.793392 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf19a86-359f-4ff7-b41b-e413952338b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f4157105f3488f807b7475008ab722', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:23.794633 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf19a86-359f-4ff7-b41b-e413952338b0 service neutron] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:23.794913 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf19a86-359f-4ff7-b41b-e413952338b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:23.796076 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf19a86-359f-4ff7-b41b-e413952338b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:23.801016 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-abf19a86-359f-4ff7-b41b-e413952338b0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:59:23.822352 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf19a86-359f-4ff7-b41b-e413952338b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:23.824994 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf19a86-359f-4ff7-b41b-e413952338b0 service neutron] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:23.836573 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b590ff61-c236-4bdd-acfd-32bc73ea5891 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a1e756ae9be4a7bac1e81acf07e8f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:23.837199 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b590ff61-c236-4bdd-acfd-32bc73ea5891 service nova] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:23.837495 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b590ff61-c236-4bdd-acfd-32bc73ea5891 service nova] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:23.837761 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b590ff61-c236-4bdd-acfd-32bc73ea5891 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:23.843278 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-b590ff61-c236-4bdd-acfd-32bc73ea5891 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:59:23.848083 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 868/1730] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 15:59:23 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:59:23.849493 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27668e19-d9f8-49bd-94b7-76d7c6b7cdd9 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:23.850079 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27668e19-d9f8-49bd-94b7-76d7c6b7cdd9 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:23.850079 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27668e19-d9f8-49bd-94b7-76d7c6b7cdd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:23.911414 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-27668e19-d9f8-49bd-94b7-76d7c6b7cdd9 None None] MFA Rules not processed for user `1a8a5ef37e0e4bb48b40e542aa376b3a`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:59:23.912643 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-27668e19-d9f8-49bd-94b7-76d7c6b7cdd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:59:23.978520 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b590ff61-c236-4bdd-acfd-32bc73ea5891 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:23.980185 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b590ff61-c236-4bdd-acfd-32bc73ea5891 service nova] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:24.012021 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 863/1731] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 15:59:23 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 437 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:59:24.013749 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55a6f3c-133b-49a1-ac12-5144dbcabe5e None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:24.022336 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55a6f3c-133b-49a1-ac12-5144dbcabe5e None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:24.035595 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55a6f3c-133b-49a1-ac12-5144dbcabe5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:24.091963 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 869/1732] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 15:59:23 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:59:24.093245 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55b3e4d-fbad-4369-9fbb-84dc6a814b15 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:24.093550 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55b3e4d-fbad-4369-9fbb-84dc6a814b15 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:24.094189 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55b3e4d-fbad-4369-9fbb-84dc6a814b15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:24.115736 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-f55a6f3c-133b-49a1-ac12-5144dbcabe5e None None] MFA Rules not processed for user `c34836a75f144f74aa3fd6a38ccb9791`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:59:24.117004 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-f55a6f3c-133b-49a1-ac12-5144dbcabe5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:59:24.134228 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-e55b3e4d-fbad-4369-9fbb-84dc6a814b15 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:59:24.139641 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-e55b3e4d-fbad-4369-9fbb-84dc6a814b15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:59:24.143427 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:59:24.256337 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 864/1733] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 15:59:24 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:59:24.266369 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778ed7d3-16b2-4535-bdb2-5d1a872a21ce None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:24.267365 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778ed7d3-16b2-4535-bdb2-5d1a872a21ce None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:24.267856 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778ed7d3-16b2-4535-bdb2-5d1a872a21ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:24.285990 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 870/1734] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 15:59:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:59:24.286914 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55b3e4d-fbad-4369-9fbb-84dc6a814b15 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:24.288348 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-e55b3e4d-fbad-4369-9fbb-84dc6a814b15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:59:24.347142 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-778ed7d3-16b2-4535-bdb2-5d1a872a21ce None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:59:24.353947 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-778ed7d3-16b2-4535-bdb2-5d1a872a21ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:59:24.368863 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 865/1735] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 15:59:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:59:24.370084 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778ed7d3-16b2-4535-bdb2-5d1a872a21ce None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:24.372473 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-778ed7d3-16b2-4535-bdb2-5d1a872a21ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:59:24.402079 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778ed7d3-16b2-4535-bdb2-5d1a872a21ce None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:24.408699 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57dd5355-e02e-43d8-a9ff-c36de45f55cf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a1e756ae9be4a7bac1e81acf07e8f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:24.409801 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dd5355-e02e-43d8-a9ff-c36de45f55cf service nova] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:24.409801 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dd5355-e02e-43d8-a9ff-c36de45f55cf service nova] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:24.410103 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dd5355-e02e-43d8-a9ff-c36de45f55cf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:24.412242 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-57dd5355-e02e-43d8-a9ff-c36de45f55cf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:59:24.426096 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57dd5355-e02e-43d8-a9ff-c36de45f55cf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:24.428073 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57dd5355-e02e-43d8-a9ff-c36de45f55cf service nova] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:24.464117 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 866/1736] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 15:59:24 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:59:24.468986 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57dd5355-e02e-43d8-a9ff-c36de45f55cf service nova] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:24.471223 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-57dd5355-e02e-43d8-a9ff-c36de45f55cf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:59:24.516200 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:59:24.523011 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55b3e4d-fbad-4369-9fbb-84dc6a814b15 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:24.527424 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57dd5355-e02e-43d8-a9ff-c36de45f55cf service nova] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:24.535444 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91814025-42d6-4a54-92a1-964101840aa4 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b70e06a7d06d4199ab4fc36342ee1d68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:24.537994 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91814025-42d6-4a54-92a1-964101840aa4 service glance] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:24.538568 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91814025-42d6-4a54-92a1-964101840aa4 service glance] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:24.539518 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91814025-42d6-4a54-92a1-964101840aa4 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:24.542605 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2872b1b-ce52-4e8d-9f5a-e47dac6a0ce4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f4157105f3488f807b7475008ab722', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:24.543472 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2872b1b-ce52-4e8d-9f5a-e47dac6a0ce4 service neutron] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:24.544353 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2872b1b-ce52-4e8d-9f5a-e47dac6a0ce4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:24.544634 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2872b1b-ce52-4e8d-9f5a-e47dac6a0ce4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:24.563782 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91814025-42d6-4a54-92a1-964101840aa4 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:24.565447 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91814025-42d6-4a54-92a1-964101840aa4 service glance] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:24.577495 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2872b1b-ce52-4e8d-9f5a-e47dac6a0ce4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:24.581801 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2872b1b-ce52-4e8d-9f5a-e47dac6a0ce4 service neutron] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:24.588741 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 871/1737] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 15:59:24 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:59:24.590539 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91814025-42d6-4a54-92a1-964101840aa4 service glance] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:24.603857 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91814025-42d6-4a54-92a1-964101840aa4 service glance] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:24.609496 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88820597-d7c7-4a68-97c9-6f136d29779f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f4157105f3488f807b7475008ab722', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:24.610121 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88820597-d7c7-4a68-97c9-6f136d29779f service neutron] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:24.610635 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88820597-d7c7-4a68-97c9-6f136d29779f service neutron] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:24.610750 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88820597-d7c7-4a68-97c9-6f136d29779f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:24.618053 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 867/1738] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 15:59:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:59:24.624598 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88820597-d7c7-4a68-97c9-6f136d29779f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:24.631419 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88820597-d7c7-4a68-97c9-6f136d29779f service neutron] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:24.652913 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 872/1739] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 15:59:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:59:25.310509 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed4cdf0-9e15-4ba3-b886-b42f9f60f818 None None] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:25.311297 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed4cdf0-9e15-4ba3-b886-b42f9f60f818 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:25.311781 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed4cdf0-9e15-4ba3-b886-b42f9f60f818 None None] PATH_INFO: `/` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:25.316473 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 868/1740] 10.222.0.20 () {54 vars in 912 bytes} [Tue Aug 30 15:59:25 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 15:59:25.322809 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7048c382-87c4-472c-8a7e-07726415a331 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:25.323690 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7048c382-87c4-472c-8a7e-07726415a331 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:25.324102 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7048c382-87c4-472c-8a7e-07726415a331 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:25.393888 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-7048c382-87c4-472c-8a7e-07726415a331 None None] MFA Rules not processed for user `65f4157105f3488f807b7475008ab722`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:59:25.398998 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-7048c382-87c4-472c-8a7e-07726415a331 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:59:25.435851 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 873/1741] 10.222.0.20 () {58 vars in 1009 bytes} [Tue Aug 30 15:59:25 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:59:25.453208 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed4cdf0-9e15-4ba3-b886-b42f9f60f818 None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:25.458478 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-bed4cdf0-9e15-4ba3-b886-b42f9f60f818 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:59:25.569995 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7048c382-87c4-472c-8a7e-07726415a331 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:25.572143 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-7048c382-87c4-472c-8a7e-07726415a331 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:59:25.612523 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7048c382-87c4-472c-8a7e-07726415a331 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:25.627144 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351f8185-bb6e-4331-862e-ac38ff89e25a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:25.628167 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351f8185-bb6e-4331-862e-ac38ff89e25a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:25.628660 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351f8185-bb6e-4331-862e-ac38ff89e25a admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:25.629117 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351f8185-bb6e-4331-862e-ac38ff89e25a admin admin] PATH_INFO: `/v3/projects/c613acae655a40a0a7abf63ea3f5990e` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:25.638115 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351f8185-bb6e-4331-862e-ac38ff89e25a admin admin] RBAC: Authorizing `identity:delete_project(project_id=c613acae655a40a0a7abf63ea3f5990e)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:25.640401 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351f8185-bb6e-4331-862e-ac38ff89e25a admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:25.710807 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed4cdf0-9e15-4ba3-b886-b42f9f60f818 None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:25.738430 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-351f8185-bb6e-4331-862e-ac38ff89e25a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c613acae655a40a0a7abf63ea3f5990e', 'request_id': 'req-351f8185-bb6e-4331-862e-ac38ff89e25a'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:59:25.762328 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-351f8185-bb6e-4331-862e-ac38ff89e25a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c613acae655a40a0a7abf63ea3f5990e', 'request_id': 'req-351f8185-bb6e-4331-862e-ac38ff89e25a'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:59:25.769746 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-351f8185-bb6e-4331-862e-ac38ff89e25a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c613acae655a40a0a7abf63ea3f5990e', 'request_id': 'req-351f8185-bb6e-4331-862e-ac38ff89e25a'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:59:25.792188 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-351f8185-bb6e-4331-862e-ac38ff89e25a admin admin] The token cache is being invalidate because project c613acae655a40a0a7abf63ea3f5990e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81511) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 15:59:25.792544 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-351f8185-bb6e-4331-862e-ac38ff89e25a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:59:25.793963 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-351f8185-bb6e-4331-862e-ac38ff89e25a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:59:25.795921 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 874/1742] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Aug 30 15:59:25 2022] DELETE /identity/v3/projects/c613acae655a40a0a7abf63ea3f5990e => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:59:25.796489 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351f8185-bb6e-4331-862e-ac38ff89e25a admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:25.802101 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-351f8185-bb6e-4331-862e-ac38ff89e25a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:59:25.848049 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af51d215-3611-4dca-b888-4df054ee3568 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc12c6d31eab4a17a160250adc2380c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:25.848490 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af51d215-3611-4dca-b888-4df054ee3568 service placement] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:25.848913 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af51d215-3611-4dca-b888-4df054ee3568 service placement] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:25.849334 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af51d215-3611-4dca-b888-4df054ee3568 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:25.852909 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-af51d215-3611-4dca-b888-4df054ee3568 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:59:25.873578 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af51d215-3611-4dca-b888-4df054ee3568 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:25.875999 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af51d215-3611-4dca-b888-4df054ee3568 service placement] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:25.896644 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 869/1743] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 15:59:25 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 444 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:59:25.897723 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af51d215-3611-4dca-b888-4df054ee3568 service placement] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:25.899208 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-af51d215-3611-4dca-b888-4df054ee3568 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:59:25.939577 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:59:25.987956 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351f8185-bb6e-4331-862e-ac38ff89e25a admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:25.993261 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2fca96-076e-469a-a50f-6b8fb64b093e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f4157105f3488f807b7475008ab722', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:25.994381 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2fca96-076e-469a-a50f-6b8fb64b093e service neutron] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:25.994808 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2fca96-076e-469a-a50f-6b8fb64b093e service neutron] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:25.995235 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2fca96-076e-469a-a50f-6b8fb64b093e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:25.997450 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-1d2fca96-076e-469a-a50f-6b8fb64b093e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:59:26.003244 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af51d215-3611-4dca-b888-4df054ee3568 service placement] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:26.010026 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a2ea34-a69b-45e8-bfc4-7003bec7994e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:26.010698 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a2ea34-a69b-45e8-bfc4-7003bec7994e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:26.010978 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a2ea34-a69b-45e8-bfc4-7003bec7994e admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:26.011248 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a2ea34-a69b-45e8-bfc4-7003bec7994e admin admin] PATH_INFO: `/v3/users/fa6c589200b840979a3ea558331786f6` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:26.012870 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a2ea34-a69b-45e8-bfc4-7003bec7994e admin admin] RBAC: Authorizing `identity:delete_user(user_id=fa6c589200b840979a3ea558331786f6)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:26.015558 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a2ea34-a69b-45e8-bfc4-7003bec7994e admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:26.018706 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2fca96-076e-469a-a50f-6b8fb64b093e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:26.024252 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2fca96-076e-469a-a50f-6b8fb64b093e service neutron] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:26.044396 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 875/1744] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 15:59:25 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 247 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:59:26.048046 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f6c378-fd9e-48d5-bf6e-bc0783a4d5b2 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:26.048442 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f6c378-fd9e-48d5-bf6e-bc0783a4d5b2 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:26.048918 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f6c378-fd9e-48d5-bf6e-bc0783a4d5b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:26.076215 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-a9f6c378-fd9e-48d5-bf6e-bc0783a4d5b2 None None] MFA Rules not processed for user `b133ae684f20468f982ae0c25020d837`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:59:26.077948 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-a9f6c378-fd9e-48d5-bf6e-bc0783a4d5b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:59:26.125094 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-14a2ea34-a69b-45e8-bfc4-7003bec7994e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fa6c589200b840979a3ea558331786f6'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:59:26.150149 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-14a2ea34-a69b-45e8-bfc4-7003bec7994e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fa6c589200b840979a3ea558331786f6'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:59:26.159268 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 876/1745] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 15:59:26 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:59:26.192194 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-14a2ea34-a69b-45e8-bfc4-7003bec7994e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fa6c589200b840979a3ea558331786f6'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:59:26.206128 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-14a2ea34-a69b-45e8-bfc4-7003bec7994e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fa6c589200b840979a3ea558331786f6'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:59:26.206947 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-14a2ea34-a69b-45e8-bfc4-7003bec7994e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fa6c589200b840979a3ea558331786f6'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:59:26.219007 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 870/1746] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Aug 30 15:59:25 2022] DELETE /identity/v3/users/fa6c589200b840979a3ea558331786f6 => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:59:26.652379 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34c6fee-2dd0-4563-9013-359a62f3265a None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:26.652972 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34c6fee-2dd0-4563-9013-359a62f3265a None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:26.653636 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34c6fee-2dd0-4563-9013-359a62f3265a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:26.705248 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:59:26.850491 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-e34c6fee-2dd0-4563-9013-359a62f3265a None None] MFA Rules not processed for user `07130d598131443486c2bc1c6673c45b`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:59:26.852799 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-e34c6fee-2dd0-4563-9013-359a62f3265a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:59:27.031727 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 877/1747] 10.222.0.20 () {58 vars in 1009 bytes} [Tue Aug 30 15:59:26 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 380 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:59:27.335010 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a2ea34-a69b-45e8-bfc4-7003bec7994e admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:27.337799 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-14a2ea34-a69b-45e8-bfc4-7003bec7994e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:59:27.590124 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34c6fee-2dd0-4563-9013-359a62f3265a None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:27.592569 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-e34c6fee-2dd0-4563-9013-359a62f3265a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:59:27.603220 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a2ea34-a69b-45e8-bfc4-7003bec7994e admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:27.608788 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0844c480-e5f6-46e1-9a20-181d78b7be52 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8fdafdc39a594bb4b0aeaf2fbc84c2fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:27.609762 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0844c480-e5f6-46e1-9a20-181d78b7be52 service swift] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:27.609762 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0844c480-e5f6-46e1-9a20-181d78b7be52 service swift] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:27.610004 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0844c480-e5f6-46e1-9a20-181d78b7be52 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:27.623940 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0844c480-e5f6-46e1-9a20-181d78b7be52 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:27.629306 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0844c480-e5f6-46e1-9a20-181d78b7be52 service swift] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:27.655718 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 871/1748] 10.222.0.20 () {60 vars in 1384 bytes} [Tue Aug 30 15:59:27 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 30 15:59:27.818431 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34c6fee-2dd0-4563-9013-359a62f3265a None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:27.837283 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1d1bcf-49a4-4a57-8039-59faa6f9e0b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:27.838022 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1d1bcf-49a4-4a57-8039-59faa6f9e0b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:27.840795 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1d1bcf-49a4-4a57-8039-59faa6f9e0b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:27.841628 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1d1bcf-49a4-4a57-8039-59faa6f9e0b2 admin admin] PATH_INFO: `/v3/projects/2e782d04ce9a493d892471610b95c474` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:27.846685 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1d1bcf-49a4-4a57-8039-59faa6f9e0b2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2e782d04ce9a493d892471610b95c474)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:27.855702 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1d1bcf-49a4-4a57-8039-59faa6f9e0b2 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:28.025951 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-4f1d1bcf-49a4-4a57-8039-59faa6f9e0b2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2e782d04ce9a493d892471610b95c474', 'request_id': 'req-4f1d1bcf-49a4-4a57-8039-59faa6f9e0b2'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:59:28.070774 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-4f1d1bcf-49a4-4a57-8039-59faa6f9e0b2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2e782d04ce9a493d892471610b95c474', 'request_id': 'req-4f1d1bcf-49a4-4a57-8039-59faa6f9e0b2'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:59:28.090673 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-4f1d1bcf-49a4-4a57-8039-59faa6f9e0b2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2e782d04ce9a493d892471610b95c474', 'request_id': 'req-4f1d1bcf-49a4-4a57-8039-59faa6f9e0b2'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:59:28.162367 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-4f1d1bcf-49a4-4a57-8039-59faa6f9e0b2 admin admin] The token cache is being invalidate because project 2e782d04ce9a493d892471610b95c474 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81511) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 15:59:28.162673 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-4f1d1bcf-49a4-4a57-8039-59faa6f9e0b2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:59:28.169829 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-4f1d1bcf-49a4-4a57-8039-59faa6f9e0b2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:59:28.175910 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 878/1749] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Aug 30 15:59:27 2022] DELETE /identity/v3/projects/2e782d04ce9a493d892471610b95c474 => generated 0 bytes in 586 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 15:59:28.182707 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcbb554-d458-4e8d-a001-bef5d1cb3e02 None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:28.183261 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcbb554-d458-4e8d-a001-bef5d1cb3e02 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:28.184134 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcbb554-d458-4e8d-a001-bef5d1cb3e02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:28.255096 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-4fcbb554-d458-4e8d-a001-bef5d1cb3e02 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:59:28.256809 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-4fcbb554-d458-4e8d-a001-bef5d1cb3e02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:59:28.272156 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:59:28.475048 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 872/1750] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 15:59:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 295 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:59:28.484680 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd916dc-e5a7-43c8-99e1-acf87d3317a3 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:28.487651 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd916dc-e5a7-43c8-99e1-acf87d3317a3 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:28.488161 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd916dc-e5a7-43c8-99e1-acf87d3317a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:28.505868 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:59:28.566698 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-9cd916dc-e5a7-43c8-99e1-acf87d3317a3 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:59:28.573098 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-9cd916dc-e5a7-43c8-99e1-acf87d3317a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:59:28.601702 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 879/1751] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 15:59:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:59:28.603177 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcbb554-d458-4e8d-a001-bef5d1cb3e02 None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:28.615605 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcbb554-d458-4e8d-a001-bef5d1cb3e02 None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:28.647510 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bf3bb7-9409-4bd4-b6b6-3d8403ab804b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:28.647837 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bf3bb7-9409-4bd4-b6b6-3d8403ab804b admin admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:28.648199 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bf3bb7-9409-4bd4-b6b6-3d8403ab804b admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:28.648470 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bf3bb7-9409-4bd4-b6b6-3d8403ab804b admin admin] PATH_INFO: `/v3/domains` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:28.650723 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42bf3bb7-9409-4bd4-b6b6-3d8403ab804b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81512) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:59:28.651165 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42bf3bb7-9409-4bd4-b6b6-3d8403ab804b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:28.653080 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42bf3bb7-9409-4bd4-b6b6-3d8403ab804b admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:28.720180 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd916dc-e5a7-43c8-99e1-acf87d3317a3 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:28.728913 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 873/1752] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Aug 30 15:59:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:59:28.734099 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd916dc-e5a7-43c8-99e1-acf87d3317a3 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:28.765475 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7de0d1-8dff-4769-9ac2-48f1d934eff8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:28.769026 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7de0d1-8dff-4769-9ac2-48f1d934eff8 admin admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:28.769026 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7de0d1-8dff-4769-9ac2-48f1d934eff8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:28.769026 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7de0d1-8dff-4769-9ac2-48f1d934eff8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:28.769897 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7de0d1-8dff-4769-9ac2-48f1d934eff8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:28.781713 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7de0d1-8dff-4769-9ac2-48f1d934eff8 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:28.842692 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 880/1753] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Aug 30 15:59:28 2022] POST /identity/v3/projects => generated 409 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 15:59:28.846842 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bf3bb7-9409-4bd4-b6b6-3d8403ab804b admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:28.857847 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bf3bb7-9409-4bd4-b6b6-3d8403ab804b admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:29.103075 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e323bc-343d-44a7-ae3d-5204f984b91e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:29.103075 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e323bc-343d-44a7-ae3d-5204f984b91e admin admin] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:29.103075 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e323bc-343d-44a7-ae3d-5204f984b91e admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:29.103075 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e323bc-343d-44a7-ae3d-5204f984b91e admin admin] PATH_INFO: `/v3/users` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:29.103917 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e323bc-343d-44a7-ae3d-5204f984b91e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:29.106158 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e323bc-343d-44a7-ae3d-5204f984b91e admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:29.146628 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d18c0cf-69ce-4a09-b7f5-dda02874e80c None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:29.147465 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d18c0cf-69ce-4a09-b7f5-dda02874e80c None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:29.147795 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d18c0cf-69ce-4a09-b7f5-dda02874e80c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:29.179270 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 874/1754] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Aug 30 15:59:28 2022] POST /identity/v3/users => generated 305 bytes in 333 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 15:59:29.181164 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4fc183-fa29-4530-b965-386312689168 None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:29.181511 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4fc183-fa29-4530-b965-386312689168 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:29.181786 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4fc183-fa29-4530-b965-386312689168 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:29.238398 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-ef4fc183-fa29-4530-b965-386312689168 None None] MFA Rules not processed for user `c34836a75f144f74aa3fd6a38ccb9791`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:59:29.240204 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-ef4fc183-fa29-4530-b965-386312689168 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:59:29.276508 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-8d18c0cf-69ce-4a09-b7f5-dda02874e80c None None] MFA Rules not processed for user `3b9a1e0efb444d40bcce2e9db2c4d347`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:59:29.277998 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-8d18c0cf-69ce-4a09-b7f5-dda02874e80c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:59:29.370655 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 875/1755] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 15:59:29 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:59:29.377918 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4fc183-fa29-4530-b965-386312689168 None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:29.417422 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4fc183-fa29-4530-b965-386312689168 None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:29.429116 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4f0d99-3fe5-44fc-a5b8-cf33440b99e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:29.429911 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4f0d99-3fe5-44fc-a5b8-cf33440b99e2 admin admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:29.430939 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4f0d99-3fe5-44fc-a5b8-cf33440b99e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:29.431055 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4f0d99-3fe5-44fc-a5b8-cf33440b99e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:29.435686 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4f0d99-3fe5-44fc-a5b8-cf33440b99e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:29.447071 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 881/1756] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 15:59:29 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 308 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:59:29.449723 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d18c0cf-69ce-4a09-b7f5-dda02874e80c None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:29.450324 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4f0d99-3fe5-44fc-a5b8-cf33440b99e2 admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:29.450675 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-8d18c0cf-69ce-4a09-b7f5-dda02874e80c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:59:29.482597 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d18c0cf-69ce-4a09-b7f5-dda02874e80c None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:29.492494 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801c2676-3d81-4654-97fa-11f2ec26240d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:29.492494 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801c2676-3d81-4654-97fa-11f2ec26240d admin admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:29.492801 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801c2676-3d81-4654-97fa-11f2ec26240d admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:29.492890 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801c2676-3d81-4654-97fa-11f2ec26240d admin admin] PATH_INFO: `/v3/projects` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:29.493560 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801c2676-3d81-4654-97fa-11f2ec26240d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:29.495496 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801c2676-3d81-4654-97fa-11f2ec26240d admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:29.508354 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 876/1757] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 30 15:59:29 2022] GET /identity/v3/roles => generated 1420 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 15:59:29.509755 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4f0d99-3fe5-44fc-a5b8-cf33440b99e2 admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:29.532902 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4f0d99-3fe5-44fc-a5b8-cf33440b99e2 admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:29.562418 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25383ba4-63ce-4421-a20b-e09e229041ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:29.563068 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25383ba4-63ce-4421-a20b-e09e229041ee admin admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:29.563347 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25383ba4-63ce-4421-a20b-e09e229041ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:29.563611 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25383ba4-63ce-4421-a20b-e09e229041ee admin admin] PATH_INFO: `/v3/roles` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:29.566474 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25383ba4-63ce-4421-a20b-e09e229041ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:29.568544 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 882/1758] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Aug 30 15:59:29 2022] POST /identity/v3/projects => generated 403 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 15:59:29.570475 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25383ba4-63ce-4421-a20b-e09e229041ee admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:29.578569 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801c2676-3d81-4654-97fa-11f2ec26240d admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:29.593275 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 877/1759] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 30 15:59:29 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 15:59:29.594463 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25383ba4-63ce-4421-a20b-e09e229041ee admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:29.616723 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25383ba4-63ce-4421-a20b-e09e229041ee admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:29.630983 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:59:29.754818 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec91ee6-6edb-443a-96a3-e844f16e9cbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:29.757759 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec91ee6-6edb-443a-96a3-e844f16e9cbe admin admin] REQUEST_METHOD: `PUT` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:29.758084 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec91ee6-6edb-443a-96a3-e844f16e9cbe admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:29.758360 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec91ee6-6edb-443a-96a3-e844f16e9cbe admin admin] PATH_INFO: `/v3/projects/7416951abd0e4c28b39210914feb63ef/users/ab323a1f8fa24fa2a01680e2e9159298/roles/df8f9f844b1b439181d75a07c1a93418` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:29.778284 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801c2676-3d81-4654-97fa-11f2ec26240d admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:29.789495 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec91ee6-6edb-443a-96a3-e844f16e9cbe admin admin] RBAC: Authorizing `identity:create_grant(project_id=7416951abd0e4c28b39210914feb63ef, user_id=ab323a1f8fa24fa2a01680e2e9159298, role_id=df8f9f844b1b439181d75a07c1a93418)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:29.793034 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec91ee6-6edb-443a-96a3-e844f16e9cbe admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:29.802836 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f496e0a3-ef03-4359-8779-4ea83b0c8649 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:29.803732 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f496e0a3-ef03-4359-8779-4ea83b0c8649 admin admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:29.804850 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f496e0a3-ef03-4359-8779-4ea83b0c8649 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:29.805989 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f496e0a3-ef03-4359-8779-4ea83b0c8649 admin admin] PATH_INFO: `/v3/users` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:29.807435 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f496e0a3-ef03-4359-8779-4ea83b0c8649 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:29.813033 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f496e0a3-ef03-4359-8779-4ea83b0c8649 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:29.822783 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 878/1760] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 30 15:59:29 2022] PUT /identity/v3/projects/7416951abd0e4c28b39210914feb63ef/users/ab323a1f8fa24fa2a01680e2e9159298/roles/df8f9f844b1b439181d75a07c1a93418 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:59:29.833252 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f278d9f-fbce-451a-ae02-0c499be6dff6 None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:29.833474 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f278d9f-fbce-451a-ae02-0c499be6dff6 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:29.833839 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f278d9f-fbce-451a-ae02-0c499be6dff6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:29.866286 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 883/1761] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Aug 30 15:59:29 2022] POST /identity/v3/users => generated 308 bytes in 289 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 15:59:29.868360 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f496e0a3-ef03-4359-8779-4ea83b0c8649 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:29.886517 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f496e0a3-ef03-4359-8779-4ea83b0c8649 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:29.889674 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-1f278d9f-fbce-451a-ae02-0c499be6dff6 None None] MFA Rules not processed for user `ab323a1f8fa24fa2a01680e2e9159298`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:59:29.891441 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-1f278d9f-fbce-451a-ae02-0c499be6dff6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:59:29.967869 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:59:30.057637 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0daeac0a-55ac-46af-8a3e-0a2a15bfcd87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:30.059000 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0daeac0a-55ac-46af-8a3e-0a2a15bfcd87 admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:30.059209 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0daeac0a-55ac-46af-8a3e-0a2a15bfcd87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:30.059512 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0daeac0a-55ac-46af-8a3e-0a2a15bfcd87 admin admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:30.063790 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0daeac0a-55ac-46af-8a3e-0a2a15bfcd87 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:30.070405 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0daeac0a-55ac-46af-8a3e-0a2a15bfcd87 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:30.080883 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 879/1762] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 15:59:29 2022] POST /identity/v3/auth/tokens => generated 3286 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:59:30.080883 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ac78c7-c398-4d68-b4fc-dfea0755f8a0 None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:30.081049 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ac78c7-c398-4d68-b4fc-dfea0755f8a0 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:30.082403 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ac78c7-c398-4d68-b4fc-dfea0755f8a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:30.109154 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 884/1763] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 30 15:59:29 2022] GET /identity/v3/roles => generated 1420 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 15:59:30.110271 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0daeac0a-55ac-46af-8a3e-0a2a15bfcd87 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:30.126427 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0daeac0a-55ac-46af-8a3e-0a2a15bfcd87 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:30.136321 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7910b23-c5be-45f0-ae5d-7c30ffe56c5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:30.137444 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7910b23-c5be-45f0-ae5d-7c30ffe56c5f admin admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:30.138526 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7910b23-c5be-45f0-ae5d-7c30ffe56c5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:30.138899 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7910b23-c5be-45f0-ae5d-7c30ffe56c5f admin admin] PATH_INFO: `/v3/projects` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:30.139623 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7910b23-c5be-45f0-ae5d-7c30ffe56c5f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:30.141648 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7910b23-c5be-45f0-ae5d-7c30ffe56c5f admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:30.172512 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-85ac78c7-c398-4d68-b4fc-dfea0755f8a0 None None] MFA Rules not processed for user `252dd6215082420cade91b100c73f100`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:59:30.174004 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-85ac78c7-c398-4d68-b4fc-dfea0755f8a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:59:30.203344 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 885/1764] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Aug 30 15:59:30 2022] POST /identity/v3/projects => generated 409 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 15:59:30.204290 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7910b23-c5be-45f0-ae5d-7c30ffe56c5f admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:30.317497 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7910b23-c5be-45f0-ae5d-7c30ffe56c5f admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:30.320131 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 880/1765] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 15:59:30 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:59:30.320982 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ac78c7-c398-4d68-b4fc-dfea0755f8a0 None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:30.333424 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ed4a43-b67c-48d0-97d0-481b4070b029 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:30.334133 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ed4a43-b67c-48d0-97d0-481b4070b029 admin admin] REQUEST_METHOD: `PUT` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:30.334412 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ed4a43-b67c-48d0-97d0-481b4070b029 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:30.334779 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ed4a43-b67c-48d0-97d0-481b4070b029 admin admin] PATH_INFO: `/v3/projects/1421ab6a8b9a4443a86efb043eb68e08/users/deb2871ae9b44eaaa5a588e5b808e5f2/roles/f265386950a641ac8413b53eda4ae34d` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:30.376591 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ac78c7-c398-4d68-b4fc-dfea0755f8a0 None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:30.378524 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ed4a43-b67c-48d0-97d0-481b4070b029 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1421ab6a8b9a4443a86efb043eb68e08, user_id=deb2871ae9b44eaaa5a588e5b808e5f2, role_id=f265386950a641ac8413b53eda4ae34d)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:30.382266 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ed4a43-b67c-48d0-97d0-481b4070b029 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:30.390393 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ed6475-8003-42b8-a8de-657ea69a72cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:30.391613 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ed6475-8003-42b8-a8de-657ea69a72cf admin admin] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:30.392420 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ed6475-8003-42b8-a8de-657ea69a72cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:30.392702 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ed6475-8003-42b8-a8de-657ea69a72cf admin admin] PATH_INFO: `/v3/users` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:30.393782 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ed6475-8003-42b8-a8de-657ea69a72cf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:30.396177 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ed6475-8003-42b8-a8de-657ea69a72cf admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:30.406288 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 886/1766] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 30 15:59:30 2022] PUT /identity/v3/projects/1421ab6a8b9a4443a86efb043eb68e08/users/deb2871ae9b44eaaa5a588e5b808e5f2/roles/f265386950a641ac8413b53eda4ae34d => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:59:30.410581 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ed4a43-b67c-48d0-97d0-481b4070b029 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:30.426551 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 881/1767] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Aug 30 15:59:30 2022] POST /identity/v3/users => generated 311 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 15:59:30.431248 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ed6475-8003-42b8-a8de-657ea69a72cf admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:30.447797 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ed6475-8003-42b8-a8de-657ea69a72cf admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:30.453874 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:59:30.487110 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ed4a43-b67c-48d0-97d0-481b4070b029 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:30.494741 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e82085-0a88-4413-9898-e0b4249522cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:30.495373 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e82085-0a88-4413-9898-e0b4249522cb admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:30.495653 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e82085-0a88-4413-9898-e0b4249522cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:30.496197 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e82085-0a88-4413-9898-e0b4249522cb admin admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:30.497019 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e82085-0a88-4413-9898-e0b4249522cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:30.499218 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e82085-0a88-4413-9898-e0b4249522cb admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:30.553622 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 887/1768] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 30 15:59:30 2022] GET /identity/v3/roles => generated 1420 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 15:59:30.556797 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e82085-0a88-4413-9898-e0b4249522cb admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:30.570660 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e82085-0a88-4413-9898-e0b4249522cb admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:30.576462 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b5e7c4-abf0-48cd-bec0-c1fcd51d46a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:30.577338 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b5e7c4-abf0-48cd-bec0-c1fcd51d46a1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:30.577621 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b5e7c4-abf0-48cd-bec0-c1fcd51d46a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:30.578722 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b5e7c4-abf0-48cd-bec0-c1fcd51d46a1 admin admin] PATH_INFO: `/v3/projects/1421ab6a8b9a4443a86efb043eb68e08/users/deb2871ae9b44eaaa5a588e5b808e5f2/roles/f265386950a641ac8413b53eda4ae34d` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:30.580913 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b5e7c4-abf0-48cd-bec0-c1fcd51d46a1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1421ab6a8b9a4443a86efb043eb68e08, user_id=deb2871ae9b44eaaa5a588e5b808e5f2, role_id=f265386950a641ac8413b53eda4ae34d)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:30.584047 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b5e7c4-abf0-48cd-bec0-c1fcd51d46a1 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:30.594952 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5099217c-e318-4af7-b2ba-ea5f360717da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:30.595691 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5099217c-e318-4af7-b2ba-ea5f360717da admin admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:30.595893 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5099217c-e318-4af7-b2ba-ea5f360717da admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:30.596185 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5099217c-e318-4af7-b2ba-ea5f360717da admin admin] PATH_INFO: `/v3/roles` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:30.596832 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5099217c-e318-4af7-b2ba-ea5f360717da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:30.601955 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5099217c-e318-4af7-b2ba-ea5f360717da admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:30.602129 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 888/1769] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 30 15:59:30 2022] PUT /identity/v3/projects/1421ab6a8b9a4443a86efb043eb68e08/users/deb2871ae9b44eaaa5a588e5b808e5f2/roles/f265386950a641ac8413b53eda4ae34d => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:59:30.614686 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 882/1770] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 30 15:59:30 2022] GET /identity/v3/roles => generated 1420 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 15:59:30.615939 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b5e7c4-abf0-48cd-bec0-c1fcd51d46a1 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:30.646534 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:59:30.766879 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b5e7c4-abf0-48cd-bec0-c1fcd51d46a1 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:30.772237 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e365b283-a1b3-46f3-a359-bd64831bffcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:30.772917 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e365b283-a1b3-46f3-a359-bd64831bffcb admin admin] REQUEST_METHOD: `PUT` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:30.773244 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e365b283-a1b3-46f3-a359-bd64831bffcb admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:30.776713 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e365b283-a1b3-46f3-a359-bd64831bffcb admin admin] PATH_INFO: `/v3/projects/e83bff629acd49d19ab681854d095ae7/users/748a08623866495aa99ee24c4c3660f5/roles/f265386950a641ac8413b53eda4ae34d` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:30.808765 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e365b283-a1b3-46f3-a359-bd64831bffcb admin admin] RBAC: Authorizing `identity:create_grant(project_id=e83bff629acd49d19ab681854d095ae7, user_id=748a08623866495aa99ee24c4c3660f5, role_id=f265386950a641ac8413b53eda4ae34d)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:30.811675 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e365b283-a1b3-46f3-a359-bd64831bffcb admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:30.876078 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 889/1771] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 30 15:59:30 2022] PUT /identity/v3/projects/e83bff629acd49d19ab681854d095ae7/users/748a08623866495aa99ee24c4c3660f5/roles/f265386950a641ac8413b53eda4ae34d => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:59:30.879643 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5099217c-e318-4af7-b2ba-ea5f360717da admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:30.893165 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5099217c-e318-4af7-b2ba-ea5f360717da admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:31.003640 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002ea0d3-07b7-4c29-aadd-3d71162deaca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:31.004321 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002ea0d3-07b7-4c29-aadd-3d71162deaca admin admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:31.004587 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002ea0d3-07b7-4c29-aadd-3d71162deaca admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:31.004865 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002ea0d3-07b7-4c29-aadd-3d71162deaca admin admin] PATH_INFO: `/v3/roles` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:31.005710 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002ea0d3-07b7-4c29-aadd-3d71162deaca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:31.008405 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002ea0d3-07b7-4c29-aadd-3d71162deaca admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:31.027152 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e365b283-a1b3-46f3-a359-bd64831bffcb admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:31.035973 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 883/1772] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 30 15:59:30 2022] GET /identity/v3/roles => generated 1420 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 15:59:31.042322 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e365b283-a1b3-46f3-a359-bd64831bffcb admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:31.049754 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b564dc-fb30-40d4-a18d-4ec1675362ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:31.050499 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b564dc-fb30-40d4-a18d-4ec1675362ec admin admin] REQUEST_METHOD: `PUT` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:31.050776 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b564dc-fb30-40d4-a18d-4ec1675362ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:31.051039 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b564dc-fb30-40d4-a18d-4ec1675362ec admin admin] PATH_INFO: `/v3/projects/e83bff629acd49d19ab681854d095ae7/users/748a08623866495aa99ee24c4c3660f5/roles/f265386950a641ac8413b53eda4ae34d` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:31.053268 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b564dc-fb30-40d4-a18d-4ec1675362ec admin admin] RBAC: Authorizing `identity:create_grant(project_id=e83bff629acd49d19ab681854d095ae7, user_id=748a08623866495aa99ee24c4c3660f5, role_id=f265386950a641ac8413b53eda4ae34d)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:31.056914 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b564dc-fb30-40d4-a18d-4ec1675362ec admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:31.079925 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 890/1773] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 30 15:59:31 2022] PUT /identity/v3/projects/e83bff629acd49d19ab681854d095ae7/users/748a08623866495aa99ee24c4c3660f5/roles/f265386950a641ac8413b53eda4ae34d => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:59:31.093426 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788a9e5c-0913-416d-a77b-37954c413ad4 None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:31.093787 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788a9e5c-0913-416d-a77b-37954c413ad4 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:31.094053 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788a9e5c-0913-416d-a77b-37954c413ad4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:31.148003 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-788a9e5c-0913-416d-a77b-37954c413ad4 None None] MFA Rules not processed for user `748a08623866495aa99ee24c4c3660f5`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:59:31.149892 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-788a9e5c-0913-416d-a77b-37954c413ad4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:59:31.313039 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 884/1774] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 15:59:31 2022] POST /identity/v3/auth/tokens => generated 3353 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:59:31.321119 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b564dc-fb30-40d4-a18d-4ec1675362ec admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:31.323787 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-11b564dc-fb30-40d4-a18d-4ec1675362ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:59:31.458181 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b564dc-fb30-40d4-a18d-4ec1675362ec admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:31.463372 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed2d204-7e79-47e6-a962-dc6792ecec16 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0100a04821804993866ea20f159663f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:31.464110 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed2d204-7e79-47e6-a962-dc6792ecec16 service cinder] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:31.464483 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed2d204-7e79-47e6-a962-dc6792ecec16 service cinder] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:31.464913 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed2d204-7e79-47e6-a962-dc6792ecec16 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:31.476556 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed2d204-7e79-47e6-a962-dc6792ecec16 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:31.478276 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed2d204-7e79-47e6-a962-dc6792ecec16 service cinder] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:31.502025 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 891/1775] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 15:59:31 2022] GET /identity/v3/auth/tokens => generated 3353 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:59:31.773649 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788a9e5c-0913-416d-a77b-37954c413ad4 None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:31.785138 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788a9e5c-0913-416d-a77b-37954c413ad4 None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:31.789422 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568b9584-3b69-4126-a5f0-59414c49ba51 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0100a04821804993866ea20f159663f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:31.790189 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568b9584-3b69-4126-a5f0-59414c49ba51 service cinder] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:31.790629 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568b9584-3b69-4126-a5f0-59414c49ba51 service cinder] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:31.791735 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568b9584-3b69-4126-a5f0-59414c49ba51 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:31.832455 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0075d79e-c482-478b-b4cd-761018686a19 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:31.832797 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0075d79e-c482-478b-b4cd-761018686a19 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:31.833130 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0075d79e-c482-478b-b4cd-761018686a19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:31.842985 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:59:31.872791 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-0075d79e-c482-478b-b4cd-761018686a19 None None] MFA Rules not processed for user `deb2871ae9b44eaaa5a588e5b808e5f2`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:59:31.874210 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-0075d79e-c482-478b-b4cd-761018686a19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:59:31.887290 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568b9584-3b69-4126-a5f0-59414c49ba51 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:31.889543 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568b9584-3b69-4126-a5f0-59414c49ba51 service cinder] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:31.906050 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 885/1776] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 15:59:31 2022] GET /identity/v3/auth/tokens => generated 3286 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:59:32.015682 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 892/1777] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 15:59:31 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:59:33.301778 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd92ce3-b9ee-46f2-b750-b9872acc080e None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:33.302818 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd92ce3-b9ee-46f2-b750-b9872acc080e None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:33.303262 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd92ce3-b9ee-46f2-b750-b9872acc080e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:33.360849 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-ddd92ce3-b9ee-46f2-b750-b9872acc080e None None] MFA Rules not processed for user `07130d598131443486c2bc1c6673c45b`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:59:33.368798 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-ddd92ce3-b9ee-46f2-b750-b9872acc080e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:59:33.421446 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841bd615-15dc-4de0-aa14-170088f34910 None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:33.421985 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841bd615-15dc-4de0-aa14-170088f34910 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:33.421985 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841bd615-15dc-4de0-aa14-170088f34910 None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:33.423314 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 893/1778] 10.222.0.20 () {54 vars in 921 bytes} [Tue Aug 30 15:59:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 15:59:33.455316 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d9b139-85e6-43d2-a1d1-a84b841ca804 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:33.455768 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d9b139-85e6-43d2-a1d1-a84b841ca804 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:33.456044 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d9b139-85e6-43d2-a1d1-a84b841ca804 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:33.508649 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-51d9b139-85e6-43d2-a1d1-a84b841ca804 None None] MFA Rules not processed for user `bc12c6d31eab4a17a160250adc2380c8`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:59:33.514017 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-51d9b139-85e6-43d2-a1d1-a84b841ca804 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:59:33.522620 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 886/1779] 10.222.0.20 () {58 vars in 1009 bytes} [Tue Aug 30 15:59:33 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:59:33.538680 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:59:33.559731 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd92ce3-b9ee-46f2-b750-b9872acc080e None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:33.563949 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-ddd92ce3-b9ee-46f2-b750-b9872acc080e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:59:33.714318 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd92ce3-b9ee-46f2-b750-b9872acc080e None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:33.720413 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b631381-b9b3-45a9-812c-ad9b296c72ee service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8fdafdc39a594bb4b0aeaf2fbc84c2fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:33.721306 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b631381-b9b3-45a9-812c-ad9b296c72ee service swift] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:33.721732 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b631381-b9b3-45a9-812c-ad9b296c72ee service swift] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:33.722142 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b631381-b9b3-45a9-812c-ad9b296c72ee service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:33.734181 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 894/1780] 10.222.0.20 () {58 vars in 1018 bytes} [Tue Aug 30 15:59:33 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 280 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:59:33.744245 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b631381-b9b3-45a9-812c-ad9b296c72ee service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:33.747041 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d9b139-85e6-43d2-a1d1-a84b841ca804 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:33.749062 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-51d9b139-85e6-43d2-a1d1-a84b841ca804 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:59:33.753754 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b631381-b9b3-45a9-812c-ad9b296c72ee service swift] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:33.768387 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d9b139-85e6-43d2-a1d1-a84b841ca804 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:33.772846 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 887/1781] 10.222.0.20 () {60 vars in 1384 bytes} [Tue Aug 30 15:59:33 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 30 15:59:33.779492 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5968d600-309d-4b39-81c3-f6c41969de31 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc12c6d31eab4a17a160250adc2380c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:33.780128 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5968d600-309d-4b39-81c3-f6c41969de31 service placement] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:33.780404 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5968d600-309d-4b39-81c3-f6c41969de31 service placement] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:33.780669 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5968d600-309d-4b39-81c3-f6c41969de31 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:33.799011 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5968d600-309d-4b39-81c3-f6c41969de31 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:33.801013 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5968d600-309d-4b39-81c3-f6c41969de31 service placement] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:33.816294 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 895/1782] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 15:59:33 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 70 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 15:59:35.275555 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b631381-b9b3-45a9-812c-ad9b296c72ee service swift] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:35.380177 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b631381-b9b3-45a9-812c-ad9b296c72ee service swift] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:35.385359 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74c54b3-799c-43b0-b74b-d23fdc51dbad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:35.386274 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74c54b3-799c-43b0-b74b-d23fdc51dbad admin admin] REQUEST_METHOD: `DELETE` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:35.386955 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74c54b3-799c-43b0-b74b-d23fdc51dbad admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:35.387373 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74c54b3-799c-43b0-b74b-d23fdc51dbad admin admin] PATH_INFO: `/v3/users/3b9a1e0efb444d40bcce2e9db2c4d347` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:35.389946 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74c54b3-799c-43b0-b74b-d23fdc51dbad admin admin] RBAC: Authorizing `identity:delete_user(user_id=3b9a1e0efb444d40bcce2e9db2c4d347)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:35.392253 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74c54b3-799c-43b0-b74b-d23fdc51dbad admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:35.394409 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:59:35.491444 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-e74c54b3-799c-43b0-b74b-d23fdc51dbad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3b9a1e0efb444d40bcce2e9db2c4d347'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:59:35.513990 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-e74c54b3-799c-43b0-b74b-d23fdc51dbad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3b9a1e0efb444d40bcce2e9db2c4d347'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:59:35.550744 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-e74c54b3-799c-43b0-b74b-d23fdc51dbad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3b9a1e0efb444d40bcce2e9db2c4d347'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:59:35.577182 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-e74c54b3-799c-43b0-b74b-d23fdc51dbad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3b9a1e0efb444d40bcce2e9db2c4d347'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:59:35.578474 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-e74c54b3-799c-43b0-b74b-d23fdc51dbad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3b9a1e0efb444d40bcce2e9db2c4d347'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:59:35.607264 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 888/1783] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Aug 30 15:59:35 2022] DELETE /identity/v3/users/3b9a1e0efb444d40bcce2e9db2c4d347 => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:59:36.140709 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5968d600-309d-4b39-81c3-f6c41969de31 service placement] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:36.166275 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5968d600-309d-4b39-81c3-f6c41969de31 service placement] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:36.282275 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22868f68-939e-4039-849e-6ac7cc9dd833 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:36.283516 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22868f68-939e-4039-849e-6ac7cc9dd833 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:36.283977 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22868f68-939e-4039-849e-6ac7cc9dd833 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:36.284581 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22868f68-939e-4039-849e-6ac7cc9dd833 admin admin] PATH_INFO: `/v3/projects/7c37c789ed174f33800e857f92cc69a8` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:36.285998 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22868f68-939e-4039-849e-6ac7cc9dd833 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7c37c789ed174f33800e857f92cc69a8)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:36.292569 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22868f68-939e-4039-849e-6ac7cc9dd833 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:36.407656 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-22868f68-939e-4039-849e-6ac7cc9dd833 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7c37c789ed174f33800e857f92cc69a8', 'request_id': 'req-22868f68-939e-4039-849e-6ac7cc9dd833'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:59:36.427210 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-22868f68-939e-4039-849e-6ac7cc9dd833 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7c37c789ed174f33800e857f92cc69a8', 'request_id': 'req-22868f68-939e-4039-849e-6ac7cc9dd833'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:59:36.441588 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-22868f68-939e-4039-849e-6ac7cc9dd833 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7c37c789ed174f33800e857f92cc69a8', 'request_id': 'req-22868f68-939e-4039-849e-6ac7cc9dd833'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:59:36.469352 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-22868f68-939e-4039-849e-6ac7cc9dd833 admin admin] The token cache is being invalidate because project 7c37c789ed174f33800e857f92cc69a8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81511) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 15:59:36.469643 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-22868f68-939e-4039-849e-6ac7cc9dd833 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:59:36.471144 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-22868f68-939e-4039-849e-6ac7cc9dd833 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:59:36.475952 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 896/1784] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Aug 30 15:59:36 2022] DELETE /identity/v3/projects/7c37c789ed174f33800e857f92cc69a8 => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:59:36.477011 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74c54b3-799c-43b0-b74b-d23fdc51dbad admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:36.479277 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-e74c54b3-799c-43b0-b74b-d23fdc51dbad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:59:36.501288 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74c54b3-799c-43b0-b74b-d23fdc51dbad admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:36.605746 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429496bf-ae56-4652-8a73-f371990d47ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:36.606567 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429496bf-ae56-4652-8a73-f371990d47ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:36.606853 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429496bf-ae56-4652-8a73-f371990d47ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:36.607126 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429496bf-ae56-4652-8a73-f371990d47ae admin admin] PATH_INFO: `/v3/users/deb2871ae9b44eaaa5a588e5b808e5f2` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:36.608304 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429496bf-ae56-4652-8a73-f371990d47ae admin admin] RBAC: Authorizing `identity:delete_user(user_id=deb2871ae9b44eaaa5a588e5b808e5f2)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:36.610325 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429496bf-ae56-4652-8a73-f371990d47ae admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:36.670194 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-429496bf-ae56-4652-8a73-f371990d47ae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'deb2871ae9b44eaaa5a588e5b808e5f2'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:59:36.682317 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-429496bf-ae56-4652-8a73-f371990d47ae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'deb2871ae9b44eaaa5a588e5b808e5f2'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:59:36.698828 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-429496bf-ae56-4652-8a73-f371990d47ae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'deb2871ae9b44eaaa5a588e5b808e5f2'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:59:36.708624 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-429496bf-ae56-4652-8a73-f371990d47ae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'deb2871ae9b44eaaa5a588e5b808e5f2'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:59:36.709635 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-429496bf-ae56-4652-8a73-f371990d47ae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'deb2871ae9b44eaaa5a588e5b808e5f2'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:59:36.711374 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:59:36.723261 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 889/1785] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Aug 30 15:59:36 2022] DELETE /identity/v3/users/deb2871ae9b44eaaa5a588e5b808e5f2 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:59:37.003582 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22868f68-939e-4039-849e-6ac7cc9dd833 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:37.006034 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-22868f68-939e-4039-849e-6ac7cc9dd833 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:59:37.033708 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22868f68-939e-4039-849e-6ac7cc9dd833 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:37.102057 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:59:37.176200 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91029bd-6579-48f6-bdfd-02dfc14d01c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:37.176902 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91029bd-6579-48f6-bdfd-02dfc14d01c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:37.177440 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91029bd-6579-48f6-bdfd-02dfc14d01c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:37.177612 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91029bd-6579-48f6-bdfd-02dfc14d01c0 admin admin] PATH_INFO: `/v3/projects/1421ab6a8b9a4443a86efb043eb68e08` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:37.179274 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91029bd-6579-48f6-bdfd-02dfc14d01c0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1421ab6a8b9a4443a86efb043eb68e08)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:37.181572 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91029bd-6579-48f6-bdfd-02dfc14d01c0 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:37.284357 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-e91029bd-6579-48f6-bdfd-02dfc14d01c0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1421ab6a8b9a4443a86efb043eb68e08', 'request_id': 'req-e91029bd-6579-48f6-bdfd-02dfc14d01c0'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:59:37.304874 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-e91029bd-6579-48f6-bdfd-02dfc14d01c0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1421ab6a8b9a4443a86efb043eb68e08', 'request_id': 'req-e91029bd-6579-48f6-bdfd-02dfc14d01c0'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:59:37.327252 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-e91029bd-6579-48f6-bdfd-02dfc14d01c0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1421ab6a8b9a4443a86efb043eb68e08', 'request_id': 'req-e91029bd-6579-48f6-bdfd-02dfc14d01c0'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:59:37.369984 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-e91029bd-6579-48f6-bdfd-02dfc14d01c0 admin admin] The token cache is being invalidate because project 1421ab6a8b9a4443a86efb043eb68e08 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81511) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 15:59:37.371377 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-e91029bd-6579-48f6-bdfd-02dfc14d01c0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:59:37.375016 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-e91029bd-6579-48f6-bdfd-02dfc14d01c0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:59:37.382745 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e949eaa-a13e-4920-8ea9-00c4981aed3f None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:37.382966 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e949eaa-a13e-4920-8ea9-00c4981aed3f None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:37.383245 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e949eaa-a13e-4920-8ea9-00c4981aed3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:37.385994 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 897/1786] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Aug 30 15:59:37 2022] DELETE /identity/v3/projects/1421ab6a8b9a4443a86efb043eb68e08 => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:59:37.427373 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-4e949eaa-a13e-4920-8ea9-00c4981aed3f None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:59:37.428625 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-4e949eaa-a13e-4920-8ea9-00c4981aed3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:59:37.510534 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 890/1787] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 15:59:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:59:37.519510 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd23171-7e42-4bdf-81af-73ba194738ca None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:37.519725 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd23171-7e42-4bdf-81af-73ba194738ca None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:37.519877 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd23171-7e42-4bdf-81af-73ba194738ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:37.550499 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-8cd23171-7e42-4bdf-81af-73ba194738ca None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:59:37.551749 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-8cd23171-7e42-4bdf-81af-73ba194738ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:59:37.563311 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 898/1788] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 15:59:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:59:37.567879 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e949eaa-a13e-4920-8ea9-00c4981aed3f None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:37.578807 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e949eaa-a13e-4920-8ea9-00c4981aed3f None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:37.583534 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b030fcad-7cf6-4d35-b2f1-616e7c167d61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:37.584149 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b030fcad-7cf6-4d35-b2f1-616e7c167d61 admin admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:37.584425 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b030fcad-7cf6-4d35-b2f1-616e7c167d61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:37.584693 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b030fcad-7cf6-4d35-b2f1-616e7c167d61 admin admin] PATH_INFO: `/v3/domains` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:37.585375 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b030fcad-7cf6-4d35-b2f1-616e7c167d61 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81512) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:59:37.585676 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b030fcad-7cf6-4d35-b2f1-616e7c167d61 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:37.587293 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b030fcad-7cf6-4d35-b2f1-616e7c167d61 admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:37.600643 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 891/1789] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Aug 30 15:59:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 15:59:37.605096 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd23171-7e42-4bdf-81af-73ba194738ca None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:37.615488 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd23171-7e42-4bdf-81af-73ba194738ca None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:37.623984 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9871e2b4-2b5b-4d07-b09c-6912497b2668 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:37.624844 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9871e2b4-2b5b-4d07-b09c-6912497b2668 admin admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:37.625352 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9871e2b4-2b5b-4d07-b09c-6912497b2668 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:37.625741 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9871e2b4-2b5b-4d07-b09c-6912497b2668 admin admin] PATH_INFO: `/v3/projects` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:37.626723 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9871e2b4-2b5b-4d07-b09c-6912497b2668 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:37.628895 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9871e2b4-2b5b-4d07-b09c-6912497b2668 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:37.665511 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 899/1790] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Aug 30 15:59:37 2022] POST /identity/v3/projects => generated 373 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 15:59:37.670213 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b030fcad-7cf6-4d35-b2f1-616e7c167d61 admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:37.684001 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b030fcad-7cf6-4d35-b2f1-616e7c167d61 admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:37.743981 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ddef81-d542-4367-b8e5-3a206a3324ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:37.744815 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ddef81-d542-4367-b8e5-3a206a3324ae admin admin] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:37.745089 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ddef81-d542-4367-b8e5-3a206a3324ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:37.745391 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ddef81-d542-4367-b8e5-3a206a3324ae admin admin] PATH_INFO: `/v3/users` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:37.746093 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ddef81-d542-4367-b8e5-3a206a3324ae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:37.748058 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ddef81-d542-4367-b8e5-3a206a3324ae admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:37.776564 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 892/1791] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Aug 30 15:59:37 2022] POST /identity/v3/users => generated 287 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 15:59:37.778097 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9871e2b4-2b5b-4d07-b09c-6912497b2668 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:37.788175 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9871e2b4-2b5b-4d07-b09c-6912497b2668 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:37.792943 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471488bb-fc43-4769-8b04-587885dc30e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:37.793586 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471488bb-fc43-4769-8b04-587885dc30e4 admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:37.793857 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471488bb-fc43-4769-8b04-587885dc30e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:37.794121 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471488bb-fc43-4769-8b04-587885dc30e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:37.794871 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471488bb-fc43-4769-8b04-587885dc30e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:37.796471 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471488bb-fc43-4769-8b04-587885dc30e4 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:37.810204 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 900/1792] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 30 15:59:37 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 15:59:37.811577 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ddef81-d542-4367-b8e5-3a206a3324ae admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:37.823709 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ddef81-d542-4367-b8e5-3a206a3324ae admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:37.828868 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93e07e6-2b26-441e-87ba-75c160673db6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:37.829617 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93e07e6-2b26-441e-87ba-75c160673db6 admin admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:37.829954 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93e07e6-2b26-441e-87ba-75c160673db6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:37.830283 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93e07e6-2b26-441e-87ba-75c160673db6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:37.830986 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93e07e6-2b26-441e-87ba-75c160673db6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:37.832776 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93e07e6-2b26-441e-87ba-75c160673db6 admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:37.844686 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 893/1793] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 30 15:59:37 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 15:59:37.849259 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471488bb-fc43-4769-8b04-587885dc30e4 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:37.859881 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471488bb-fc43-4769-8b04-587885dc30e4 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:37.864793 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8e989d-b9c8-4f0e-a0b1-bfa69cb983b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:37.865489 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8e989d-b9c8-4f0e-a0b1-bfa69cb983b8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:37.865764 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8e989d-b9c8-4f0e-a0b1-bfa69cb983b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:37.866057 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8e989d-b9c8-4f0e-a0b1-bfa69cb983b8 admin admin] PATH_INFO: `/v3/projects/b43a2b4fe41c4d50a3c588a9c2473d63/users/d28800d13b6049f5952cf40a19f6dc22/roles/df8f9f844b1b439181d75a07c1a93418` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:37.881521 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8e989d-b9c8-4f0e-a0b1-bfa69cb983b8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b43a2b4fe41c4d50a3c588a9c2473d63, user_id=d28800d13b6049f5952cf40a19f6dc22, role_id=df8f9f844b1b439181d75a07c1a93418)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:37.884184 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8e989d-b9c8-4f0e-a0b1-bfa69cb983b8 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:37.899776 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 901/1794] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 30 15:59:37 2022] PUT /identity/v3/projects/b43a2b4fe41c4d50a3c588a9c2473d63/users/d28800d13b6049f5952cf40a19f6dc22/roles/df8f9f844b1b439181d75a07c1a93418 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 15:59:37.911904 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5325806-d6c9-4a11-8856-2e4620b4d1b8 None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:37.912192 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5325806-d6c9-4a11-8856-2e4620b4d1b8 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:37.912558 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5325806-d6c9-4a11-8856-2e4620b4d1b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:37.946509 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-f5325806-d6c9-4a11-8856-2e4620b4d1b8 None None] MFA Rules not processed for user `d28800d13b6049f5952cf40a19f6dc22`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:59:37.947749 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-f5325806-d6c9-4a11-8856-2e4620b4d1b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:59:38.008464 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:59:38.021495 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 894/1795] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 15:59:37 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:59:38.025799 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8e989d-b9c8-4f0e-a0b1-bfa69cb983b8 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:38.110050 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:59:38.130781 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8e989d-b9c8-4f0e-a0b1-bfa69cb983b8 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:38.142245 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c547413-6060-42c8-aba4-55f76ab8eacf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:38.143149 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c547413-6060-42c8-aba4-55f76ab8eacf admin admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:38.148831 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c547413-6060-42c8-aba4-55f76ab8eacf admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:38.157042 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c547413-6060-42c8-aba4-55f76ab8eacf admin admin] PATH_INFO: `/v3/projects` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:38.158083 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c547413-6060-42c8-aba4-55f76ab8eacf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:38.193427 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c547413-6060-42c8-aba4-55f76ab8eacf admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:38.240776 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 902/1796] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Aug 30 15:59:38 2022] POST /identity/v3/projects => generated 371 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 15:59:38.246180 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5325806-d6c9-4a11-8856-2e4620b4d1b8 None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:38.265504 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5325806-d6c9-4a11-8856-2e4620b4d1b8 None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:38.447838 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ccfa39-bcc4-4bd7-8405-26f3a1af0395 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:38.447838 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ccfa39-bcc4-4bd7-8405-26f3a1af0395 admin admin] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:38.447838 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ccfa39-bcc4-4bd7-8405-26f3a1af0395 admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:38.447838 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ccfa39-bcc4-4bd7-8405-26f3a1af0395 admin admin] PATH_INFO: `/v3/users` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:38.448379 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ccfa39-bcc4-4bd7-8405-26f3a1af0395 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:38.449943 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ccfa39-bcc4-4bd7-8405-26f3a1af0395 admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:38.524252 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c547413-6060-42c8-aba4-55f76ab8eacf admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:38.544308 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 895/1797] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Aug 30 15:59:38 2022] POST /identity/v3/users => generated 292 bytes in 298 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 15:59:38.548873 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c547413-6060-42c8-aba4-55f76ab8eacf admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:38.563701 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfd529f-5dc8-4e89-a8f7-a5457adda1f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:38.569881 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfd529f-5dc8-4e89-a8f7-a5457adda1f3 admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:38.570669 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfd529f-5dc8-4e89-a8f7-a5457adda1f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:38.571370 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfd529f-5dc8-4e89-a8f7-a5457adda1f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:38.572895 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfd529f-5dc8-4e89-a8f7-a5457adda1f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:38.578406 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfd529f-5dc8-4e89-a8f7-a5457adda1f3 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:38.606853 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 903/1798] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 30 15:59:38 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 15:59:38.610865 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ccfa39-bcc4-4bd7-8405-26f3a1af0395 admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:38.633796 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ccfa39-bcc4-4bd7-8405-26f3a1af0395 admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:38.663897 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f15299-76dd-4aaa-a695-e8438457c497 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:38.665015 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f15299-76dd-4aaa-a695-e8438457c497 admin admin] REQUEST_METHOD: `PUT` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:38.665346 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f15299-76dd-4aaa-a695-e8438457c497 admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:38.665624 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f15299-76dd-4aaa-a695-e8438457c497 admin admin] PATH_INFO: `/v3/projects/d3a4652d80eb450a801d6aa788d94d01/users/e6373234a7984f01aeb25925e90a0c85/roles/f265386950a641ac8413b53eda4ae34d` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:38.706199 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f15299-76dd-4aaa-a695-e8438457c497 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d3a4652d80eb450a801d6aa788d94d01, user_id=e6373234a7984f01aeb25925e90a0c85, role_id=f265386950a641ac8413b53eda4ae34d)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:38.713076 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f15299-76dd-4aaa-a695-e8438457c497 admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:38.759067 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfd529f-5dc8-4e89-a8f7-a5457adda1f3 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:38.761300 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 896/1799] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 30 15:59:38 2022] PUT /identity/v3/projects/d3a4652d80eb450a801d6aa788d94d01/users/e6373234a7984f01aeb25925e90a0c85/roles/f265386950a641ac8413b53eda4ae34d => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:59:38.781575 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfd529f-5dc8-4e89-a8f7-a5457adda1f3 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:38.935985 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a5c1e7-1a97-47f1-9617-e8159ce95dd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:38.937545 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a5c1e7-1a97-47f1-9617-e8159ce95dd3 admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:38.941298 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a5c1e7-1a97-47f1-9617-e8159ce95dd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:38.941670 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a5c1e7-1a97-47f1-9617-e8159ce95dd3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:38.942508 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a5c1e7-1a97-47f1-9617-e8159ce95dd3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:38.946272 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a5c1e7-1a97-47f1-9617-e8159ce95dd3 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:38.960474 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 904/1800] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 30 15:59:38 2022] GET /identity/v3/roles => generated 1420 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 15:59:38.961774 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f15299-76dd-4aaa-a695-e8438457c497 admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:38.976431 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f15299-76dd-4aaa-a695-e8438457c497 admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:38.982880 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436e1db9-e181-4b6e-a987-a72fb6283cb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:38.983912 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436e1db9-e181-4b6e-a987-a72fb6283cb0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:38.984254 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436e1db9-e181-4b6e-a987-a72fb6283cb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:38.984718 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436e1db9-e181-4b6e-a987-a72fb6283cb0 admin admin] PATH_INFO: `/v3/projects/d3a4652d80eb450a801d6aa788d94d01/users/e6373234a7984f01aeb25925e90a0c85/roles/f265386950a641ac8413b53eda4ae34d` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:38.987080 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436e1db9-e181-4b6e-a987-a72fb6283cb0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d3a4652d80eb450a801d6aa788d94d01, user_id=e6373234a7984f01aeb25925e90a0c85, role_id=f265386950a641ac8413b53eda4ae34d)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:38.990797 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436e1db9-e181-4b6e-a987-a72fb6283cb0 admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:39.018486 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 897/1801] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 30 15:59:38 2022] PUT /identity/v3/projects/d3a4652d80eb450a801d6aa788d94d01/users/e6373234a7984f01aeb25925e90a0c85/roles/f265386950a641ac8413b53eda4ae34d => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:59:39.020255 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6227cb46-71cf-46dd-ac36-b4ae30a74136 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:39.020605 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6227cb46-71cf-46dd-ac36-b4ae30a74136 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:39.020948 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6227cb46-71cf-46dd-ac36-b4ae30a74136 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:39.096366 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-6227cb46-71cf-46dd-ac36-b4ae30a74136 None None] MFA Rules not processed for user `e6373234a7984f01aeb25925e90a0c85`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:59:39.097897 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-6227cb46-71cf-46dd-ac36-b4ae30a74136 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:59:39.199992 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 905/1802] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 15:59:39 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:59:39.258277 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436e1db9-e181-4b6e-a987-a72fb6283cb0 admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:39.260032 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-436e1db9-e181-4b6e-a987-a72fb6283cb0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:59:39.342184 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436e1db9-e181-4b6e-a987-a72fb6283cb0 admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:39.346144 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e17e4a3-81bf-4f3a-bca7-c26b83e04a93 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0100a04821804993866ea20f159663f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:39.346847 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e17e4a3-81bf-4f3a-bca7-c26b83e04a93 service cinder] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:39.347125 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e17e4a3-81bf-4f3a-bca7-c26b83e04a93 service cinder] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:39.347395 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e17e4a3-81bf-4f3a-bca7-c26b83e04a93 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:39.357713 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e17e4a3-81bf-4f3a-bca7-c26b83e04a93 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:39.359219 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e17e4a3-81bf-4f3a-bca7-c26b83e04a93 service cinder] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:39.375244 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 898/1803] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 15:59:39 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:59:39.486243 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6227cb46-71cf-46dd-ac36-b4ae30a74136 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:39.496740 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6227cb46-71cf-46dd-ac36-b4ae30a74136 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:39.500814 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac37a1e7-afb8-4cbc-86b9-d3f7862f7d31 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0100a04821804993866ea20f159663f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:39.501468 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac37a1e7-afb8-4cbc-86b9-d3f7862f7d31 service cinder] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:39.501748 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac37a1e7-afb8-4cbc-86b9-d3f7862f7d31 service cinder] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:39.502018 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac37a1e7-afb8-4cbc-86b9-d3f7862f7d31 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:39.548746 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:59:39.563579 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac37a1e7-afb8-4cbc-86b9-d3f7862f7d31 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:39.565615 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac37a1e7-afb8-4cbc-86b9-d3f7862f7d31 service cinder] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:39.575441 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 906/1804] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 15:59:39 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:59:43.077346 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2a0eaa-3d84-48ae-a129-68b5816f38aa None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:43.077946 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2a0eaa-3d84-48ae-a129-68b5816f38aa None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:43.078114 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2a0eaa-3d84-48ae-a129-68b5816f38aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:43.102911 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-5c2a0eaa-3d84-48ae-a129-68b5816f38aa None None] MFA Rules not processed for user `65f4157105f3488f807b7475008ab722`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:59:43.104401 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-5c2a0eaa-3d84-48ae-a129-68b5816f38aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:59:43.243514 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:59:43.261198 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 899/1805] 10.222.0.42 () {58 vars in 1015 bytes} [Tue Aug 30 15:59:43 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:59:43.743178 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac37a1e7-afb8-4cbc-86b9-d3f7862f7d31 service cinder] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:43.744622 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-ac37a1e7-afb8-4cbc-86b9-d3f7862f7d31 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:59:43.810146 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac37a1e7-afb8-4cbc-86b9-d3f7862f7d31 service cinder] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:43.814512 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c35ed35-7555-43f9-ad10-d96350ea78e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:43.815175 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c35ed35-7555-43f9-ad10-d96350ea78e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:43.815448 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c35ed35-7555-43f9-ad10-d96350ea78e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:43.815714 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c35ed35-7555-43f9-ad10-d96350ea78e5 admin admin] PATH_INFO: `/v3/users/ab323a1f8fa24fa2a01680e2e9159298` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:43.816821 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c35ed35-7555-43f9-ad10-d96350ea78e5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ab323a1f8fa24fa2a01680e2e9159298)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:43.818756 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c35ed35-7555-43f9-ad10-d96350ea78e5 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:43.868086 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-5c35ed35-7555-43f9-ad10-d96350ea78e5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ab323a1f8fa24fa2a01680e2e9159298'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:59:43.879395 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-5c35ed35-7555-43f9-ad10-d96350ea78e5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ab323a1f8fa24fa2a01680e2e9159298'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:59:43.887579 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-5c35ed35-7555-43f9-ad10-d96350ea78e5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ab323a1f8fa24fa2a01680e2e9159298'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:59:43.888220 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-5c35ed35-7555-43f9-ad10-d96350ea78e5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ab323a1f8fa24fa2a01680e2e9159298'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:59:43.898804 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-5c35ed35-7555-43f9-ad10-d96350ea78e5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ab323a1f8fa24fa2a01680e2e9159298'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:59:43.908552 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 907/1806] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Aug 30 15:59:43 2022] DELETE /identity/v3/users/ab323a1f8fa24fa2a01680e2e9159298 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:59:43.914996 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff37253a-adad-459d-ae39-86c3df5c18d2 None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:43.915345 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff37253a-adad-459d-ae39-86c3df5c18d2 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:43.915565 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff37253a-adad-459d-ae39-86c3df5c18d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:43.936720 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-ff37253a-adad-459d-ae39-86c3df5c18d2 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:59:43.937951 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-ff37253a-adad-459d-ae39-86c3df5c18d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:59:43.999024 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 900/1807] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 15:59:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:59:44.009141 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c35ed35-7555-43f9-ad10-d96350ea78e5 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:44.010687 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-5c35ed35-7555-43f9-ad10-d96350ea78e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:59:44.068786 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c35ed35-7555-43f9-ad10-d96350ea78e5 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:44.072767 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8e25d6-7501-49cb-b53b-cfb8ea639054 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f4157105f3488f807b7475008ab722', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:44.073452 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8e25d6-7501-49cb-b53b-cfb8ea639054 service neutron] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:44.073766 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8e25d6-7501-49cb-b53b-cfb8ea639054 service neutron] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:44.074058 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8e25d6-7501-49cb-b53b-cfb8ea639054 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:44.083327 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8e25d6-7501-49cb-b53b-cfb8ea639054 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:44.084759 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8e25d6-7501-49cb-b53b-cfb8ea639054 service neutron] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:44.094636 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 908/1808] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 15:59:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:59:44.370873 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff37253a-adad-459d-ae39-86c3df5c18d2 None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:44.372623 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-ff37253a-adad-459d-ae39-86c3df5c18d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:59:44.384908 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff37253a-adad-459d-ae39-86c3df5c18d2 None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:44.390017 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ace9c8-4783-41e0-9caa-1afb92a672b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:44.390696 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ace9c8-4783-41e0-9caa-1afb92a672b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:44.391022 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ace9c8-4783-41e0-9caa-1afb92a672b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:44.391293 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ace9c8-4783-41e0-9caa-1afb92a672b8 admin admin] PATH_INFO: `/v3/projects/7416951abd0e4c28b39210914feb63ef` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:44.392674 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ace9c8-4783-41e0-9caa-1afb92a672b8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7416951abd0e4c28b39210914feb63ef)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:44.394623 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ace9c8-4783-41e0-9caa-1afb92a672b8 admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:44.444914 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-30ace9c8-4783-41e0-9caa-1afb92a672b8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7416951abd0e4c28b39210914feb63ef', 'request_id': 'req-30ace9c8-4783-41e0-9caa-1afb92a672b8'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:59:44.461333 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-30ace9c8-4783-41e0-9caa-1afb92a672b8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7416951abd0e4c28b39210914feb63ef', 'request_id': 'req-30ace9c8-4783-41e0-9caa-1afb92a672b8'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:59:44.468684 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-30ace9c8-4783-41e0-9caa-1afb92a672b8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7416951abd0e4c28b39210914feb63ef', 'request_id': 'req-30ace9c8-4783-41e0-9caa-1afb92a672b8'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:59:44.479528 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-30ace9c8-4783-41e0-9caa-1afb92a672b8 admin admin] The token cache is being invalidate because project 7416951abd0e4c28b39210914feb63ef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81512) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 15:59:44.479820 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-30ace9c8-4783-41e0-9caa-1afb92a672b8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:59:44.480520 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-30ace9c8-4783-41e0-9caa-1afb92a672b8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:59:44.482107 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 901/1809] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Aug 30 15:59:44 2022] DELETE /identity/v3/projects/7416951abd0e4c28b39210914feb63ef => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:59:44.486183 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8e25d6-7501-49cb-b53b-cfb8ea639054 service neutron] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:44.498242 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8e25d6-7501-49cb-b53b-cfb8ea639054 service neutron] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:44.504237 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:59:44.568437 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba2dbb1-0d4b-4963-a92b-4936748199c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:44.569072 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba2dbb1-0d4b-4963-a92b-4936748199c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:44.569381 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba2dbb1-0d4b-4963-a92b-4936748199c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:44.569653 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba2dbb1-0d4b-4963-a92b-4936748199c2 admin admin] PATH_INFO: `/v3/users/748a08623866495aa99ee24c4c3660f5` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:44.570760 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba2dbb1-0d4b-4963-a92b-4936748199c2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=748a08623866495aa99ee24c4c3660f5)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:44.572686 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba2dbb1-0d4b-4963-a92b-4936748199c2 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:44.621616 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-7ba2dbb1-0d4b-4963-a92b-4936748199c2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '748a08623866495aa99ee24c4c3660f5'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:59:44.636044 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-7ba2dbb1-0d4b-4963-a92b-4936748199c2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '748a08623866495aa99ee24c4c3660f5'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:59:44.646909 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-7ba2dbb1-0d4b-4963-a92b-4936748199c2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '748a08623866495aa99ee24c4c3660f5'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:59:44.647855 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-7ba2dbb1-0d4b-4963-a92b-4936748199c2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '748a08623866495aa99ee24c4c3660f5'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:59:44.654726 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-7ba2dbb1-0d4b-4963-a92b-4936748199c2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '748a08623866495aa99ee24c4c3660f5'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:59:44.660780 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 909/1810] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Aug 30 15:59:44 2022] DELETE /identity/v3/users/748a08623866495aa99ee24c4c3660f5 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:59:45.006892 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ace9c8-4783-41e0-9caa-1afb92a672b8 admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:45.008465 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-30ace9c8-4783-41e0-9caa-1afb92a672b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:59:45.022115 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ace9c8-4783-41e0-9caa-1afb92a672b8 admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:45.079951 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2a79db-6255-473a-82f1-076944924ec0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:45.080589 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2a79db-6255-473a-82f1-076944924ec0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:45.080863 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2a79db-6255-473a-82f1-076944924ec0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:45.081131 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2a79db-6255-473a-82f1-076944924ec0 admin admin] PATH_INFO: `/v3/projects/e83bff629acd49d19ab681854d095ae7` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:45.082387 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2a79db-6255-473a-82f1-076944924ec0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e83bff629acd49d19ab681854d095ae7)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:45.084315 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2a79db-6255-473a-82f1-076944924ec0 admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:45.110144 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:59:45.138475 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-6e2a79db-6255-473a-82f1-076944924ec0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e83bff629acd49d19ab681854d095ae7', 'request_id': 'req-6e2a79db-6255-473a-82f1-076944924ec0'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:59:45.148004 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-6e2a79db-6255-473a-82f1-076944924ec0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e83bff629acd49d19ab681854d095ae7', 'request_id': 'req-6e2a79db-6255-473a-82f1-076944924ec0'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:59:45.154623 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-6e2a79db-6255-473a-82f1-076944924ec0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e83bff629acd49d19ab681854d095ae7', 'request_id': 'req-6e2a79db-6255-473a-82f1-076944924ec0'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:59:45.165901 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-6e2a79db-6255-473a-82f1-076944924ec0 admin admin] The token cache is being invalidate because project e83bff629acd49d19ab681854d095ae7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81512) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 15:59:45.166192 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-6e2a79db-6255-473a-82f1-076944924ec0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:59:45.166928 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-6e2a79db-6255-473a-82f1-076944924ec0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:59:45.168556 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 902/1811] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Aug 30 15:59:45 2022] DELETE /identity/v3/projects/e83bff629acd49d19ab681854d095ae7 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:59:45.173770 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2413d6b7-6b8e-4a80-be1e-39e15a67d394 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:45.174230 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2413d6b7-6b8e-4a80-be1e-39e15a67d394 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:45.174328 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2413d6b7-6b8e-4a80-be1e-39e15a67d394 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:45.194363 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-2413d6b7-6b8e-4a80-be1e-39e15a67d394 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:59:45.195569 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-2413d6b7-6b8e-4a80-be1e-39e15a67d394 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:59:45.254029 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 910/1812] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 15:59:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:59:45.262057 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce639e9-835b-4ad0-bcf4-a10d93172aed None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:45.262340 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce639e9-835b-4ad0-bcf4-a10d93172aed None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:45.262633 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce639e9-835b-4ad0-bcf4-a10d93172aed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:45.282429 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-bce639e9-835b-4ad0-bcf4-a10d93172aed None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:59:45.283620 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-bce639e9-835b-4ad0-bcf4-a10d93172aed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:59:45.292144 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 903/1813] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 15:59:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:59:45.298602 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2413d6b7-6b8e-4a80-be1e-39e15a67d394 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:45.308453 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2413d6b7-6b8e-4a80-be1e-39e15a67d394 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:45.313858 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bbe6c5-9caa-4af8-b5af-0328027dfd81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:45.314456 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bbe6c5-9caa-4af8-b5af-0328027dfd81 admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:45.314723 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bbe6c5-9caa-4af8-b5af-0328027dfd81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:45.314991 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bbe6c5-9caa-4af8-b5af-0328027dfd81 admin admin] PATH_INFO: `/v3/domains` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:45.315659 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01bbe6c5-9caa-4af8-b5af-0328027dfd81 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:59:45.315962 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01bbe6c5-9caa-4af8-b5af-0328027dfd81 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:45.317590 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01bbe6c5-9caa-4af8-b5af-0328027dfd81 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:45.329691 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 911/1814] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Aug 30 15:59:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:59:45.334058 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce639e9-835b-4ad0-bcf4-a10d93172aed None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:45.343449 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce639e9-835b-4ad0-bcf4-a10d93172aed None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:45.348843 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120f5c86-ac31-4190-8c0c-6abd9ea5fa2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:45.349478 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120f5c86-ac31-4190-8c0c-6abd9ea5fa2d admin admin] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:45.349757 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120f5c86-ac31-4190-8c0c-6abd9ea5fa2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:45.350033 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120f5c86-ac31-4190-8c0c-6abd9ea5fa2d admin admin] PATH_INFO: `/v3/projects` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:45.350693 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120f5c86-ac31-4190-8c0c-6abd9ea5fa2d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:45.352594 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120f5c86-ac31-4190-8c0c-6abd9ea5fa2d admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:45.384826 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 904/1815] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Aug 30 15:59:45 2022] POST /identity/v3/projects => generated 385 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 15:59:45.389150 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bbe6c5-9caa-4af8-b5af-0328027dfd81 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:45.399564 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bbe6c5-9caa-4af8-b5af-0328027dfd81 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:45.409838 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:59:45.461785 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2373ecb1-28fd-4e98-9f07-cd579b4a0bad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:45.462397 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2373ecb1-28fd-4e98-9f07-cd579b4a0bad admin admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:45.462684 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2373ecb1-28fd-4e98-9f07-cd579b4a0bad admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:45.462954 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2373ecb1-28fd-4e98-9f07-cd579b4a0bad admin admin] PATH_INFO: `/v3/users` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:45.463599 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2373ecb1-28fd-4e98-9f07-cd579b4a0bad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:45.465554 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2373ecb1-28fd-4e98-9f07-cd579b4a0bad admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:45.489511 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 912/1816] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Aug 30 15:59:45 2022] POST /identity/v3/users => generated 293 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 15:59:45.493909 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120f5c86-ac31-4190-8c0c-6abd9ea5fa2d admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:45.506019 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120f5c86-ac31-4190-8c0c-6abd9ea5fa2d admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:45.511073 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba53d70-8347-4c7c-880f-0d518777c5a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:45.511735 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba53d70-8347-4c7c-880f-0d518777c5a8 admin admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:45.512010 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba53d70-8347-4c7c-880f-0d518777c5a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:45.512294 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba53d70-8347-4c7c-880f-0d518777c5a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:45.513013 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba53d70-8347-4c7c-880f-0d518777c5a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:45.514664 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba53d70-8347-4c7c-880f-0d518777c5a8 admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:45.525817 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 905/1817] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 30 15:59:45 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 15:59:45.529903 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2373ecb1-28fd-4e98-9f07-cd579b4a0bad admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:45.539805 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2373ecb1-28fd-4e98-9f07-cd579b4a0bad admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:45.544513 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9936c8ed-6c46-4104-9e4d-6fb9a1c3e2ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:45.545125 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9936c8ed-6c46-4104-9e4d-6fb9a1c3e2ea admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:45.545485 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9936c8ed-6c46-4104-9e4d-6fb9a1c3e2ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:45.545757 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9936c8ed-6c46-4104-9e4d-6fb9a1c3e2ea admin admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:45.546405 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9936c8ed-6c46-4104-9e4d-6fb9a1c3e2ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:45.550858 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9936c8ed-6c46-4104-9e4d-6fb9a1c3e2ea admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:45.567262 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 913/1818] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 30 15:59:45 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 15:59:45.568041 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba53d70-8347-4c7c-880f-0d518777c5a8 admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:45.580940 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba53d70-8347-4c7c-880f-0d518777c5a8 admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:45.585558 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f0e203-1437-4bbb-aebf-206b8b0299a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:45.586321 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f0e203-1437-4bbb-aebf-206b8b0299a7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:45.586661 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f0e203-1437-4bbb-aebf-206b8b0299a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:45.587014 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f0e203-1437-4bbb-aebf-206b8b0299a7 admin admin] PATH_INFO: `/v3/projects/bca836f0a23f49ae84534786cf8a3d3a/users/fc5536e9207643aaade044805d93b02c/roles/df8f9f844b1b439181d75a07c1a93418` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:45.606455 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f0e203-1437-4bbb-aebf-206b8b0299a7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bca836f0a23f49ae84534786cf8a3d3a, user_id=fc5536e9207643aaade044805d93b02c, role_id=df8f9f844b1b439181d75a07c1a93418)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:45.609513 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f0e203-1437-4bbb-aebf-206b8b0299a7 admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:45.624645 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 906/1819] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 30 15:59:45 2022] PUT /identity/v3/projects/bca836f0a23f49ae84534786cf8a3d3a/users/fc5536e9207643aaade044805d93b02c/roles/df8f9f844b1b439181d75a07c1a93418 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:59:45.635252 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c6e3a2-67d1-460e-bdd3-d9891c2c3ca0 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:45.635618 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c6e3a2-67d1-460e-bdd3-d9891c2c3ca0 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:45.635957 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c6e3a2-67d1-460e-bdd3-d9891c2c3ca0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:45.674373 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-96c6e3a2-67d1-460e-bdd3-d9891c2c3ca0 None None] MFA Rules not processed for user `fc5536e9207643aaade044805d93b02c`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:59:45.675588 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-96c6e3a2-67d1-460e-bdd3-d9891c2c3ca0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:59:45.756330 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 914/1820] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 15:59:45 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:59:45.763540 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f0e203-1437-4bbb-aebf-206b8b0299a7 admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:45.846104 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f0e203-1437-4bbb-aebf-206b8b0299a7 admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:45.850697 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244d1596-2515-445a-ae19-3fef184070f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:45.851324 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244d1596-2515-445a-ae19-3fef184070f8 admin admin] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:45.851577 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244d1596-2515-445a-ae19-3fef184070f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:45.851843 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244d1596-2515-445a-ae19-3fef184070f8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:45.852464 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244d1596-2515-445a-ae19-3fef184070f8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:45.854452 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244d1596-2515-445a-ae19-3fef184070f8 admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:45.885033 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 907/1821] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Aug 30 15:59:45 2022] POST /identity/v3/projects => generated 383 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 15:59:45.889321 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c6e3a2-67d1-460e-bdd3-d9891c2c3ca0 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:45.900380 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c6e3a2-67d1-460e-bdd3-d9891c2c3ca0 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:45.963137 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3404a5e-ca99-44cb-84a2-4619942bb9ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:45.964318 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3404a5e-ca99-44cb-84a2-4619942bb9ed admin admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:45.964318 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3404a5e-ca99-44cb-84a2-4619942bb9ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:45.964318 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3404a5e-ca99-44cb-84a2-4619942bb9ed admin admin] PATH_INFO: `/v3/users` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:45.965376 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3404a5e-ca99-44cb-84a2-4619942bb9ed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:45.967489 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3404a5e-ca99-44cb-84a2-4619942bb9ed admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:46.001341 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 915/1822] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Aug 30 15:59:45 2022] POST /identity/v3/users => generated 298 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 15:59:46.005470 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244d1596-2515-445a-ae19-3fef184070f8 admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:46.019695 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244d1596-2515-445a-ae19-3fef184070f8 admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:46.027572 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9c6a1d-c0e5-44a4-a842-fceff13750ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:46.028445 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9c6a1d-c0e5-44a4-a842-fceff13750ed admin admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:46.028863 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9c6a1d-c0e5-44a4-a842-fceff13750ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:46.029323 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9c6a1d-c0e5-44a4-a842-fceff13750ed admin admin] PATH_INFO: `/v3/roles` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:46.030114 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9c6a1d-c0e5-44a4-a842-fceff13750ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:46.032361 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9c6a1d-c0e5-44a4-a842-fceff13750ed admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:46.052708 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 908/1823] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 30 15:59:46 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 15:59:46.053920 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3404a5e-ca99-44cb-84a2-4619942bb9ed admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:46.076755 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3404a5e-ca99-44cb-84a2-4619942bb9ed admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:46.083290 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5918b8e0-7e08-44ae-b680-ba3d65061b4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:46.084468 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5918b8e0-7e08-44ae-b680-ba3d65061b4b admin admin] REQUEST_METHOD: `PUT` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:46.084915 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5918b8e0-7e08-44ae-b680-ba3d65061b4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:46.085411 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5918b8e0-7e08-44ae-b680-ba3d65061b4b admin admin] PATH_INFO: `/v3/projects/2c48e49e3f0c483ca3a022f0a1ce9fa1/users/4b1e1a8b7d3441ae8bfe3ca15eb0607d/roles/f265386950a641ac8413b53eda4ae34d` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:46.095860 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:59:46.107474 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5918b8e0-7e08-44ae-b680-ba3d65061b4b admin admin] RBAC: Authorizing `identity:create_grant(project_id=2c48e49e3f0c483ca3a022f0a1ce9fa1, user_id=4b1e1a8b7d3441ae8bfe3ca15eb0607d, role_id=f265386950a641ac8413b53eda4ae34d)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:46.115734 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5918b8e0-7e08-44ae-b680-ba3d65061b4b admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:46.147899 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 916/1824] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 30 15:59:46 2022] PUT /identity/v3/projects/2c48e49e3f0c483ca3a022f0a1ce9fa1/users/4b1e1a8b7d3441ae8bfe3ca15eb0607d/roles/f265386950a641ac8413b53eda4ae34d => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 15:59:46.148446 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9c6a1d-c0e5-44a4-a842-fceff13750ed admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:46.150469 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-aa9c6a1d-c0e5-44a4-a842-fceff13750ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:59:46.160118 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5918b8e0-7e08-44ae-b680-ba3d65061b4b admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:46.214200 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=90,keystone:DELETE=6,keystone:INSERT=4 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:59:46.298147 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9c6a1d-c0e5-44a4-a842-fceff13750ed admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:46.298535 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5918b8e0-7e08-44ae-b680-ba3d65061b4b admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:46.303746 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1059c956-9105-4afa-a27b-3665bd46041c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:46.304385 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1059c956-9105-4afa-a27b-3665bd46041c admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:46.304663 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1059c956-9105-4afa-a27b-3665bd46041c admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:46.304932 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1059c956-9105-4afa-a27b-3665bd46041c admin admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:46.305629 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1059c956-9105-4afa-a27b-3665bd46041c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:46.312041 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1059c956-9105-4afa-a27b-3665bd46041c admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:46.324208 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 917/1825] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 30 15:59:46 2022] GET /identity/v3/roles => generated 1420 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 15:59:46.328931 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1059c956-9105-4afa-a27b-3665bd46041c admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:46.344875 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1059c956-9105-4afa-a27b-3665bd46041c admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:46.350147 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d321759-8499-439b-9b44-9d5a09a9056b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:46.350829 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d321759-8499-439b-9b44-9d5a09a9056b admin admin] REQUEST_METHOD: `PUT` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:46.351107 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d321759-8499-439b-9b44-9d5a09a9056b admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:46.351394 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d321759-8499-439b-9b44-9d5a09a9056b admin admin] PATH_INFO: `/v3/projects/2c48e49e3f0c483ca3a022f0a1ce9fa1/users/4b1e1a8b7d3441ae8bfe3ca15eb0607d/roles/f265386950a641ac8413b53eda4ae34d` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:46.353427 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d321759-8499-439b-9b44-9d5a09a9056b admin admin] RBAC: Authorizing `identity:create_grant(project_id=2c48e49e3f0c483ca3a022f0a1ce9fa1, user_id=4b1e1a8b7d3441ae8bfe3ca15eb0607d, role_id=f265386950a641ac8413b53eda4ae34d)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:46.356163 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d321759-8499-439b-9b44-9d5a09a9056b admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:46.369180 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 918/1826] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 30 15:59:46 2022] PUT /identity/v3/projects/2c48e49e3f0c483ca3a022f0a1ce9fa1/users/4b1e1a8b7d3441ae8bfe3ca15eb0607d/roles/f265386950a641ac8413b53eda4ae34d => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:59:46.380572 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e303448-9688-49f6-8399-f70db31fb578 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:46.380909 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e303448-9688-49f6-8399-f70db31fb578 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:46.381130 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e303448-9688-49f6-8399-f70db31fb578 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:46.384314 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a18051-b340-4c26-9551-debb84f3146b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f4157105f3488f807b7475008ab722', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:46.385437 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a18051-b340-4c26-9551-debb84f3146b service neutron] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:46.386535 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a18051-b340-4c26-9551-debb84f3146b service neutron] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:46.387025 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a18051-b340-4c26-9551-debb84f3146b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:46.389650 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-95a18051-b340-4c26-9551-debb84f3146b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:59:46.401886 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a18051-b340-4c26-9551-debb84f3146b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:46.403691 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a18051-b340-4c26-9551-debb84f3146b service neutron] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:46.416378 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 909/1827] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 15:59:46 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 268 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 15:59:46.418464 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-0e303448-9688-49f6-8399-f70db31fb578 None None] MFA Rules not processed for user `4b1e1a8b7d3441ae8bfe3ca15eb0607d`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:59:46.421855 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-0e303448-9688-49f6-8399-f70db31fb578 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:59:46.514317 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 919/1828] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 15:59:46 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 15:59:46.525981 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a18051-b340-4c26-9551-debb84f3146b service neutron] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:46.527468 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-95a18051-b340-4c26-9551-debb84f3146b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:59:46.587654 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a18051-b340-4c26-9551-debb84f3146b service neutron] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:46.657357 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1318430-3561-43e8-8e68-a832d30ee5f8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0100a04821804993866ea20f159663f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:46.658110 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1318430-3561-43e8-8e68-a832d30ee5f8 service cinder] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:46.658451 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1318430-3561-43e8-8e68-a832d30ee5f8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:46.658826 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1318430-3561-43e8-8e68-a832d30ee5f8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:46.671283 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1318430-3561-43e8-8e68-a832d30ee5f8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:46.672743 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1318430-3561-43e8-8e68-a832d30ee5f8 service cinder] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:46.683830 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 910/1829] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 15:59:46 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:59:47.452971 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e303448-9688-49f6-8399-f70db31fb578 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:47.514571 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:59:47.520996 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e303448-9688-49f6-8399-f70db31fb578 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:47.527038 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffad6662-2780-4ca0-ab44-7c99f2c8e52a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:47.527682 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffad6662-2780-4ca0-ab44-7c99f2c8e52a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:47.527960 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffad6662-2780-4ca0-ab44-7c99f2c8e52a admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:47.528254 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffad6662-2780-4ca0-ab44-7c99f2c8e52a admin admin] PATH_INFO: `/v3/users/fc5536e9207643aaade044805d93b02c` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:47.529941 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffad6662-2780-4ca0-ab44-7c99f2c8e52a admin admin] RBAC: Authorizing `identity:delete_user(user_id=fc5536e9207643aaade044805d93b02c)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:47.532025 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffad6662-2780-4ca0-ab44-7c99f2c8e52a admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:47.585594 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-ffad6662-2780-4ca0-ab44-7c99f2c8e52a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fc5536e9207643aaade044805d93b02c'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:59:47.599570 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-ffad6662-2780-4ca0-ab44-7c99f2c8e52a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fc5536e9207643aaade044805d93b02c'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:59:47.609967 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-ffad6662-2780-4ca0-ab44-7c99f2c8e52a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fc5536e9207643aaade044805d93b02c'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:59:47.610776 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-ffad6662-2780-4ca0-ab44-7c99f2c8e52a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fc5536e9207643aaade044805d93b02c'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:59:47.627245 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-ffad6662-2780-4ca0-ab44-7c99f2c8e52a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fc5536e9207643aaade044805d93b02c'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:59:47.636087 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 920/1830] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Aug 30 15:59:47 2022] DELETE /identity/v3/users/fc5536e9207643aaade044805d93b02c => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:59:47.641089 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f662b3ac-3297-4d3a-a7c0-f36d7cd96bb1 None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:47.641633 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f662b3ac-3297-4d3a-a7c0-f36d7cd96bb1 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:47.641717 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f662b3ac-3297-4d3a-a7c0-f36d7cd96bb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:47.661238 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-f662b3ac-3297-4d3a-a7c0-f36d7cd96bb1 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:59:47.662451 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-f662b3ac-3297-4d3a-a7c0-f36d7cd96bb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:59:47.701512 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:59:47.741410 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 911/1831] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 15:59:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:59:47.750604 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffad6662-2780-4ca0-ab44-7c99f2c8e52a admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:47.752418 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-ffad6662-2780-4ca0-ab44-7c99f2c8e52a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:59:47.872868 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffad6662-2780-4ca0-ab44-7c99f2c8e52a admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:47.878083 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f758202-3bbf-407b-8af7-42b84abcba19 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f4157105f3488f807b7475008ab722', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:47.879167 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f758202-3bbf-407b-8af7-42b84abcba19 service neutron] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:47.879453 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f758202-3bbf-407b-8af7-42b84abcba19 service neutron] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:47.879780 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f758202-3bbf-407b-8af7-42b84abcba19 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:47.893461 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f758202-3bbf-407b-8af7-42b84abcba19 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:47.894981 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f758202-3bbf-407b-8af7-42b84abcba19 service neutron] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:47.906176 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 921/1832] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 15:59:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:59:48.050896 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3c0f0e-a4ed-42b2-b8f1-b2316f2c306f None None] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:48.051551 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3c0f0e-a4ed-42b2-b8f1-b2316f2c306f None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:48.052051 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3c0f0e-a4ed-42b2-b8f1-b2316f2c306f None None] PATH_INFO: `/` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:48.055509 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 912/1833] 10.222.0.20 () {54 vars in 921 bytes} [Tue Aug 30 15:59:48 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 15:59:48.078933 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de4a267-a76a-4731-9d54-a03b12642e45 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:48.079266 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de4a267-a76a-4731-9d54-a03b12642e45 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:48.079608 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de4a267-a76a-4731-9d54-a03b12642e45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:48.138291 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-3de4a267-a76a-4731-9d54-a03b12642e45 None None] MFA Rules not processed for user `7a1e756ae9be4a7bac1e81acf07e8f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:59:48.142182 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-3de4a267-a76a-4731-9d54-a03b12642e45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:59:48.281449 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 922/1834] 10.222.0.20 () {58 vars in 1018 bytes} [Tue Aug 30 15:59:48 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:59:48.318347 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3c0f0e-a4ed-42b2-b8f1-b2316f2c306f None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:48.320412 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-ee3c0f0e-a4ed-42b2-b8f1-b2316f2c306f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:59:48.352436 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3c0f0e-a4ed-42b2-b8f1-b2316f2c306f None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:48.357534 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e466faf-b458-43f0-b078-2f73747b8107 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:48.358179 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e466faf-b458-43f0-b078-2f73747b8107 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:48.358466 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e466faf-b458-43f0-b078-2f73747b8107 admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:48.358740 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e466faf-b458-43f0-b078-2f73747b8107 admin admin] PATH_INFO: `/v3/projects/bca836f0a23f49ae84534786cf8a3d3a` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:48.359871 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e466faf-b458-43f0-b078-2f73747b8107 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bca836f0a23f49ae84534786cf8a3d3a)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:48.361880 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e466faf-b458-43f0-b078-2f73747b8107 admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:48.423725 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-8e466faf-b458-43f0-b078-2f73747b8107 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bca836f0a23f49ae84534786cf8a3d3a', 'request_id': 'req-8e466faf-b458-43f0-b078-2f73747b8107'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:59:48.426579 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de4a267-a76a-4731-9d54-a03b12642e45 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:48.428362 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-3de4a267-a76a-4731-9d54-a03b12642e45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:59:48.448553 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-8e466faf-b458-43f0-b078-2f73747b8107 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bca836f0a23f49ae84534786cf8a3d3a', 'request_id': 'req-8e466faf-b458-43f0-b078-2f73747b8107'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:59:48.449970 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de4a267-a76a-4731-9d54-a03b12642e45 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:48.456343 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-8e466faf-b458-43f0-b078-2f73747b8107 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bca836f0a23f49ae84534786cf8a3d3a', 'request_id': 'req-8e466faf-b458-43f0-b078-2f73747b8107'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:59:48.456820 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:59:48.469810 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-8e466faf-b458-43f0-b078-2f73747b8107 admin admin] The token cache is being invalidate because project bca836f0a23f49ae84534786cf8a3d3a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81512) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 15:59:48.470090 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-8e466faf-b458-43f0-b078-2f73747b8107 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:59:48.471088 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-8e466faf-b458-43f0-b078-2f73747b8107 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:59:48.475780 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 913/1835] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Aug 30 15:59:48 2022] DELETE /identity/v3/projects/bca836f0a23f49ae84534786cf8a3d3a => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:59:48.478352 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e466faf-b458-43f0-b078-2f73747b8107 admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:48.480010 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-8e466faf-b458-43f0-b078-2f73747b8107 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:59:48.544892 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b9082e-a27e-4e6d-b1e8-f09d5a9e1819 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a1e756ae9be4a7bac1e81acf07e8f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:48.545881 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b9082e-a27e-4e6d-b1e8-f09d5a9e1819 service nova] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:48.546229 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b9082e-a27e-4e6d-b1e8-f09d5a9e1819 service nova] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:48.546794 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b9082e-a27e-4e6d-b1e8-f09d5a9e1819 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:48.549376 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-b6b9082e-a27e-4e6d-b1e8-f09d5a9e1819 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:59:48.562617 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b9082e-a27e-4e6d-b1e8-f09d5a9e1819 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:48.563662 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e466faf-b458-43f0-b078-2f73747b8107 admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:48.564478 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b9082e-a27e-4e6d-b1e8-f09d5a9e1819 service nova] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:48.568573 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a2deb1-b2f9-4daf-ae55-79a738dcc67a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:48.569236 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a2deb1-b2f9-4daf-ae55-79a738dcc67a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:48.569537 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a2deb1-b2f9-4daf-ae55-79a738dcc67a admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:48.569812 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a2deb1-b2f9-4daf-ae55-79a738dcc67a admin admin] PATH_INFO: `/v3/users/4b1e1a8b7d3441ae8bfe3ca15eb0607d` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:48.571054 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a2deb1-b2f9-4daf-ae55-79a738dcc67a admin admin] RBAC: Authorizing `identity:delete_user(user_id=4b1e1a8b7d3441ae8bfe3ca15eb0607d)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:48.574259 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a2deb1-b2f9-4daf-ae55-79a738dcc67a admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:48.576963 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 923/1836] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 15:59:48 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:59:48.646867 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-27a2deb1-b2f9-4daf-ae55-79a738dcc67a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4b1e1a8b7d3441ae8bfe3ca15eb0607d'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:59:48.658458 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-27a2deb1-b2f9-4daf-ae55-79a738dcc67a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4b1e1a8b7d3441ae8bfe3ca15eb0607d'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:59:48.670339 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:59:48.678687 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-27a2deb1-b2f9-4daf-ae55-79a738dcc67a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4b1e1a8b7d3441ae8bfe3ca15eb0607d'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:59:48.686922 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-27a2deb1-b2f9-4daf-ae55-79a738dcc67a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4b1e1a8b7d3441ae8bfe3ca15eb0607d'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:59:48.687664 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-27a2deb1-b2f9-4daf-ae55-79a738dcc67a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4b1e1a8b7d3441ae8bfe3ca15eb0607d'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:59:48.694374 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 914/1837] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Aug 30 15:59:48 2022] DELETE /identity/v3/users/4b1e1a8b7d3441ae8bfe3ca15eb0607d => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 15:59:49.065586 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b9082e-a27e-4e6d-b1e8-f09d5a9e1819 service nova] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:49.081866 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b9082e-a27e-4e6d-b1e8-f09d5a9e1819 service nova] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:49.175494 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252551ef-6a7a-4fae-9b0b-aba9aa028c66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:49.176168 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252551ef-6a7a-4fae-9b0b-aba9aa028c66 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:49.176450 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252551ef-6a7a-4fae-9b0b-aba9aa028c66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:49.176719 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252551ef-6a7a-4fae-9b0b-aba9aa028c66 admin admin] PATH_INFO: `/v3/projects/2c48e49e3f0c483ca3a022f0a1ce9fa1` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:49.177949 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252551ef-6a7a-4fae-9b0b-aba9aa028c66 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2c48e49e3f0c483ca3a022f0a1ce9fa1)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:49.180302 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252551ef-6a7a-4fae-9b0b-aba9aa028c66 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:49.249816 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-252551ef-6a7a-4fae-9b0b-aba9aa028c66 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2c48e49e3f0c483ca3a022f0a1ce9fa1', 'request_id': 'req-252551ef-6a7a-4fae-9b0b-aba9aa028c66'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:59:49.261376 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-252551ef-6a7a-4fae-9b0b-aba9aa028c66 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2c48e49e3f0c483ca3a022f0a1ce9fa1', 'request_id': 'req-252551ef-6a7a-4fae-9b0b-aba9aa028c66'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:59:49.272773 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-252551ef-6a7a-4fae-9b0b-aba9aa028c66 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2c48e49e3f0c483ca3a022f0a1ce9fa1', 'request_id': 'req-252551ef-6a7a-4fae-9b0b-aba9aa028c66'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:59:49.289108 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-252551ef-6a7a-4fae-9b0b-aba9aa028c66 admin admin] The token cache is being invalidate because project 2c48e49e3f0c483ca3a022f0a1ce9fa1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81511) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 15:59:49.289551 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-252551ef-6a7a-4fae-9b0b-aba9aa028c66 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:59:49.290239 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-252551ef-6a7a-4fae-9b0b-aba9aa028c66 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:59:49.296232 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 924/1838] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Aug 30 15:59:49 2022] DELETE /identity/v3/projects/2c48e49e3f0c483ca3a022f0a1ce9fa1 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:59:49.298152 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589fd713-b135-4ce0-acb6-81361a464533 None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:49.298661 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589fd713-b135-4ce0-acb6-81361a464533 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:49.299149 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589fd713-b135-4ce0-acb6-81361a464533 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:49.322774 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-589fd713-b135-4ce0-acb6-81361a464533 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:59:49.324219 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-589fd713-b135-4ce0-acb6-81361a464533 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:59:49.434797 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 915/1839] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 15:59:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:59:49.441164 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41fada4-507c-416f-a2ce-196b1ec07776 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:49.441797 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41fada4-507c-416f-a2ce-196b1ec07776 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:49.444844 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41fada4-507c-416f-a2ce-196b1ec07776 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:49.471712 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-d41fada4-507c-416f-a2ce-196b1ec07776 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:59:49.473279 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-d41fada4-507c-416f-a2ce-196b1ec07776 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:59:49.486105 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 925/1840] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 15:59:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:59:49.491304 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589fd713-b135-4ce0-acb6-81361a464533 None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:49.504151 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589fd713-b135-4ce0-acb6-81361a464533 None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:49.514881 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf8ce87-acea-49be-ac36-980b8165c4a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:49.515581 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf8ce87-acea-49be-ac36-980b8165c4a9 admin admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:49.515866 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf8ce87-acea-49be-ac36-980b8165c4a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:49.516136 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf8ce87-acea-49be-ac36-980b8165c4a9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:49.517242 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf8ce87-acea-49be-ac36-980b8165c4a9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81512) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:59:49.517767 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf8ce87-acea-49be-ac36-980b8165c4a9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:49.521155 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf8ce87-acea-49be-ac36-980b8165c4a9 admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:49.540417 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 916/1841] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Aug 30 15:59:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 15:59:49.543639 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41fada4-507c-416f-a2ce-196b1ec07776 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:49.556734 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41fada4-507c-416f-a2ce-196b1ec07776 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:49.567137 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c3f720-6d3a-47ff-982b-6fce9924ab5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:49.568059 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c3f720-6d3a-47ff-982b-6fce9924ab5d admin admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:49.568424 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c3f720-6d3a-47ff-982b-6fce9924ab5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:49.568764 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c3f720-6d3a-47ff-982b-6fce9924ab5d admin admin] PATH_INFO: `/v3/projects` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:49.569794 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c3f720-6d3a-47ff-982b-6fce9924ab5d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:49.572522 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c3f720-6d3a-47ff-982b-6fce9924ab5d admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:49.605908 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:59:49.630854 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 926/1842] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Aug 30 15:59:49 2022] POST /identity/v3/projects => generated 371 bytes in 85 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 15:59:49.633014 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf8ce87-acea-49be-ac36-980b8165c4a9 admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:49.650583 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf8ce87-acea-49be-ac36-980b8165c4a9 admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:49.723568 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7ab2aa-36aa-4c4e-96d8-e03c739a6033 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:49.725122 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7ab2aa-36aa-4c4e-96d8-e03c739a6033 admin admin] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:49.725728 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7ab2aa-36aa-4c4e-96d8-e03c739a6033 admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:49.726609 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7ab2aa-36aa-4c4e-96d8-e03c739a6033 admin admin] PATH_INFO: `/v3/users` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:49.727943 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7ab2aa-36aa-4c4e-96d8-e03c739a6033 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:49.730259 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7ab2aa-36aa-4c4e-96d8-e03c739a6033 admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:49.766354 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 917/1843] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Aug 30 15:59:49 2022] POST /identity/v3/users => generated 286 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 15:59:49.767087 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c3f720-6d3a-47ff-982b-6fce9924ab5d admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:49.795182 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c3f720-6d3a-47ff-982b-6fce9924ab5d admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:49.809883 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94449516-cdb2-4ac3-8deb-a1a85105fa9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:49.810526 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94449516-cdb2-4ac3-8deb-a1a85105fa9b admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:49.810798 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94449516-cdb2-4ac3-8deb-a1a85105fa9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:49.811064 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94449516-cdb2-4ac3-8deb-a1a85105fa9b admin admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:49.811713 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94449516-cdb2-4ac3-8deb-a1a85105fa9b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:49.813495 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94449516-cdb2-4ac3-8deb-a1a85105fa9b admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:49.853993 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 927/1844] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 30 15:59:49 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 15:59:49.858656 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7ab2aa-36aa-4c4e-96d8-e03c739a6033 admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:49.875123 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7ab2aa-36aa-4c4e-96d8-e03c739a6033 admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:49.888082 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690831dd-adf6-4266-abf1-ca56ada38498 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:49.888847 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690831dd-adf6-4266-abf1-ca56ada38498 admin admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:49.890491 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690831dd-adf6-4266-abf1-ca56ada38498 admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:49.891483 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690831dd-adf6-4266-abf1-ca56ada38498 admin admin] PATH_INFO: `/v3/roles` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:49.893091 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690831dd-adf6-4266-abf1-ca56ada38498 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:49.899905 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690831dd-adf6-4266-abf1-ca56ada38498 admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:49.941645 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 918/1845] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 30 15:59:49 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 15:59:49.942913 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94449516-cdb2-4ac3-8deb-a1a85105fa9b admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:49.961894 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94449516-cdb2-4ac3-8deb-a1a85105fa9b admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:49.977681 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1756d27c-2ad6-45e8-8e57-895e55c91435 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:49.978505 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1756d27c-2ad6-45e8-8e57-895e55c91435 admin admin] REQUEST_METHOD: `PUT` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:49.980211 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1756d27c-2ad6-45e8-8e57-895e55c91435 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:49.980847 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1756d27c-2ad6-45e8-8e57-895e55c91435 admin admin] PATH_INFO: `/v3/projects/eebb52a011cb44b1930952b4c9a181ec/users/64c270098b4b47aca93591aac7c0b244/roles/df8f9f844b1b439181d75a07c1a93418` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:50.010679 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1756d27c-2ad6-45e8-8e57-895e55c91435 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eebb52a011cb44b1930952b4c9a181ec, user_id=64c270098b4b47aca93591aac7c0b244, role_id=df8f9f844b1b439181d75a07c1a93418)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:50.014577 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1756d27c-2ad6-45e8-8e57-895e55c91435 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:50.035807 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 928/1846] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 30 15:59:49 2022] PUT /identity/v3/projects/eebb52a011cb44b1930952b4c9a181ec/users/64c270098b4b47aca93591aac7c0b244/roles/df8f9f844b1b439181d75a07c1a93418 => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:59:50.046159 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadeef6b-9fae-4dfb-ae97-702b676d36af None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:50.046343 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadeef6b-9fae-4dfb-ae97-702b676d36af None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:50.047015 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadeef6b-9fae-4dfb-ae97-702b676d36af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:50.124252 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-cadeef6b-9fae-4dfb-ae97-702b676d36af None None] MFA Rules not processed for user `64c270098b4b47aca93591aac7c0b244`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:59:50.129472 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-cadeef6b-9fae-4dfb-ae97-702b676d36af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:59:50.242138 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:59:50.300865 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 919/1847] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 15:59:50 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:59:50.313136 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1756d27c-2ad6-45e8-8e57-895e55c91435 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:50.316848 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-1756d27c-2ad6-45e8-8e57-895e55c91435 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:59:50.459056 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1756d27c-2ad6-45e8-8e57-895e55c91435 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:50.472954 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4575a08b-42c9-40c9-bc38-2e925091cd86 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0100a04821804993866ea20f159663f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:50.476436 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4575a08b-42c9-40c9-bc38-2e925091cd86 service cinder] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:50.476929 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4575a08b-42c9-40c9-bc38-2e925091cd86 service cinder] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:50.478695 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4575a08b-42c9-40c9-bc38-2e925091cd86 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:50.504076 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4575a08b-42c9-40c9-bc38-2e925091cd86 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:50.505864 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4575a08b-42c9-40c9-bc38-2e925091cd86 service cinder] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:50.539794 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 929/1848] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 15:59:50 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:59:50.570966 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadeef6b-9fae-4dfb-ae97-702b676d36af None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:50.702786 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadeef6b-9fae-4dfb-ae97-702b676d36af None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:50.725438 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14539137-7f65-49d9-8a45-a4657d270e1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:50.726526 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14539137-7f65-49d9-8a45-a4657d270e1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:50.726959 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14539137-7f65-49d9-8a45-a4657d270e1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:50.727360 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14539137-7f65-49d9-8a45-a4657d270e1c admin admin] PATH_INFO: `/v3/users/64c270098b4b47aca93591aac7c0b244` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:50.730001 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14539137-7f65-49d9-8a45-a4657d270e1c admin admin] RBAC: Authorizing `identity:delete_user(user_id=64c270098b4b47aca93591aac7c0b244)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:50.732735 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14539137-7f65-49d9-8a45-a4657d270e1c admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:50.811014 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-14539137-7f65-49d9-8a45-a4657d270e1c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '64c270098b4b47aca93591aac7c0b244'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:59:50.842552 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-14539137-7f65-49d9-8a45-a4657d270e1c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '64c270098b4b47aca93591aac7c0b244'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:59:50.871364 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-14539137-7f65-49d9-8a45-a4657d270e1c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '64c270098b4b47aca93591aac7c0b244'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:59:50.894701 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-14539137-7f65-49d9-8a45-a4657d270e1c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '64c270098b4b47aca93591aac7c0b244'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:59:50.895832 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-14539137-7f65-49d9-8a45-a4657d270e1c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '64c270098b4b47aca93591aac7c0b244'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:59:50.906500 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 920/1849] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Aug 30 15:59:50 2022] DELETE /identity/v3/users/64c270098b4b47aca93591aac7c0b244 => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:59:50.911733 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e055bf4-2d90-4248-9315-50ec7ba63069 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:50.911998 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e055bf4-2d90-4248-9315-50ec7ba63069 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:50.912509 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e055bf4-2d90-4248-9315-50ec7ba63069 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:50.955789 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-3e055bf4-2d90-4248-9315-50ec7ba63069 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:59:50.981479 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-3e055bf4-2d90-4248-9315-50ec7ba63069 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:59:51.203533 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 930/1850] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 15:59:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 294 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:59:51.234468 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14539137-7f65-49d9-8a45-a4657d270e1c admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:51.236362 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-14539137-7f65-49d9-8a45-a4657d270e1c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:59:51.324707 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14539137-7f65-49d9-8a45-a4657d270e1c admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:51.329488 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869cc272-c2b2-427e-82c3-99b124c28723 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f4157105f3488f807b7475008ab722', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:51.330350 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869cc272-c2b2-427e-82c3-99b124c28723 service neutron] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:51.330798 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869cc272-c2b2-427e-82c3-99b124c28723 service neutron] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:51.331350 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869cc272-c2b2-427e-82c3-99b124c28723 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:51.344292 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869cc272-c2b2-427e-82c3-99b124c28723 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:51.346226 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869cc272-c2b2-427e-82c3-99b124c28723 service neutron] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:51.352195 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:59:51.358696 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 921/1851] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 15:59:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:59:52.522564 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e055bf4-2d90-4248-9315-50ec7ba63069 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:52.524108 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-3e055bf4-2d90-4248-9315-50ec7ba63069 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:59:52.539698 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e055bf4-2d90-4248-9315-50ec7ba63069 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:52.547456 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4ef10c-b52d-48cf-87a1-b20ed0a1c336 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:52.548529 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4ef10c-b52d-48cf-87a1-b20ed0a1c336 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:52.549161 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4ef10c-b52d-48cf-87a1-b20ed0a1c336 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:52.549831 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4ef10c-b52d-48cf-87a1-b20ed0a1c336 admin admin] PATH_INFO: `/v3/users/1a8a2a49f97346589cedadae04903627` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:52.551779 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4ef10c-b52d-48cf-87a1-b20ed0a1c336 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1a8a2a49f97346589cedadae04903627)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:52.554391 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4ef10c-b52d-48cf-87a1-b20ed0a1c336 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:52.633454 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-2d4ef10c-b52d-48cf-87a1-b20ed0a1c336 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1a8a2a49f97346589cedadae04903627'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:59:52.650341 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869cc272-c2b2-427e-82c3-99b124c28723 service neutron] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:52.652422 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=91,keystone:INSERT=1,keystone:DELETE=8 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:59:52.652882 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-869cc272-c2b2-427e-82c3-99b124c28723 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:59:52.657023 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-2d4ef10c-b52d-48cf-87a1-b20ed0a1c336 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1a8a2a49f97346589cedadae04903627'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:59:52.669914 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-2d4ef10c-b52d-48cf-87a1-b20ed0a1c336 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1a8a2a49f97346589cedadae04903627'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:59:52.671494 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-2d4ef10c-b52d-48cf-87a1-b20ed0a1c336 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1a8a2a49f97346589cedadae04903627'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:59:52.683034 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869cc272-c2b2-427e-82c3-99b124c28723 service neutron] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:52.697588 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29901ce7-15ab-4681-aee3-bf0824ce5585 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:52.698529 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29901ce7-15ab-4681-aee3-bf0824ce5585 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:52.699719 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-2d4ef10c-b52d-48cf-87a1-b20ed0a1c336 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1a8a2a49f97346589cedadae04903627'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:59:52.700078 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29901ce7-15ab-4681-aee3-bf0824ce5585 admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:52.700668 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29901ce7-15ab-4681-aee3-bf0824ce5585 admin admin] PATH_INFO: `/v3/projects/eebb52a011cb44b1930952b4c9a181ec` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:52.702909 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29901ce7-15ab-4681-aee3-bf0824ce5585 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eebb52a011cb44b1930952b4c9a181ec)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:52.704942 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29901ce7-15ab-4681-aee3-bf0824ce5585 admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:52.718510 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 931/1852] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Aug 30 15:59:52 2022] DELETE /identity/v3/users/1a8a2a49f97346589cedadae04903627 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:59:52.723078 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e14c678-8998-434b-bda2-70e7caa6c599 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:52.723562 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e14c678-8998-434b-bda2-70e7caa6c599 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:52.723999 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e14c678-8998-434b-bda2-70e7caa6c599 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:52.790932 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-4e14c678-8998-434b-bda2-70e7caa6c599 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:59:52.792619 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-4e14c678-8998-434b-bda2-70e7caa6c599 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:59:52.831950 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-29901ce7-15ab-4681-aee3-bf0824ce5585 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eebb52a011cb44b1930952b4c9a181ec', 'request_id': 'req-29901ce7-15ab-4681-aee3-bf0824ce5585'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:59:52.859823 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-29901ce7-15ab-4681-aee3-bf0824ce5585 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eebb52a011cb44b1930952b4c9a181ec', 'request_id': 'req-29901ce7-15ab-4681-aee3-bf0824ce5585'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:59:52.871763 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-29901ce7-15ab-4681-aee3-bf0824ce5585 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eebb52a011cb44b1930952b4c9a181ec', 'request_id': 'req-29901ce7-15ab-4681-aee3-bf0824ce5585'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:59:52.900118 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-29901ce7-15ab-4681-aee3-bf0824ce5585 admin admin] The token cache is being invalidate because project eebb52a011cb44b1930952b4c9a181ec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81512) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 15:59:52.900415 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-29901ce7-15ab-4681-aee3-bf0824ce5585 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:59:52.902664 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-29901ce7-15ab-4681-aee3-bf0824ce5585 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:59:52.905644 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 922/1853] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Aug 30 15:59:52 2022] DELETE /identity/v3/projects/eebb52a011cb44b1930952b4c9a181ec => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:59:52.922904 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d9ca17-6875-44ec-b7a4-ac6fd7adcb38 None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:52.923992 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d9ca17-6875-44ec-b7a4-ac6fd7adcb38 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:52.925639 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d9ca17-6875-44ec-b7a4-ac6fd7adcb38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:52.943214 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 932/1854] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 15:59:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 15:59:52.945815 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ee9372-41cf-4381-ba3b-d3676f0cfec4 None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:52.946290 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ee9372-41cf-4381-ba3b-d3676f0cfec4 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:52.946734 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ee9372-41cf-4381-ba3b-d3676f0cfec4 None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:52.949053 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 933/1855] 10.222.0.20 () {54 vars in 921 bytes} [Tue Aug 30 15:59:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 15:59:52.956992 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-66d9ca17-6875-44ec-b7a4-ac6fd7adcb38 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:59:52.958395 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-66d9ca17-6875-44ec-b7a4-ac6fd7adcb38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:59:52.971146 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ee9372-41cf-4381-ba3b-d3676f0cfec4 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:52.973932 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-a0ee9372-41cf-4381-ba3b-d3676f0cfec4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:59:52.981371 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 923/1856] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 15:59:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:59:52.981371 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e547cc4-0efc-41c7-96a1-6bed3a8a9e01 None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:52.981371 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e547cc4-0efc-41c7-96a1-6bed3a8a9e01 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:52.981371 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e547cc4-0efc-41c7-96a1-6bed3a8a9e01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:53.027531 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-5e547cc4-0efc-41c7-96a1-6bed3a8a9e01 None None] MFA Rules not processed for user `7a1e756ae9be4a7bac1e81acf07e8f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:59:53.029694 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-5e547cc4-0efc-41c7-96a1-6bed3a8a9e01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:59:53.045806 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ee9372-41cf-4381-ba3b-d3676f0cfec4 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:53.051467 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b6194f-150b-4616-aae5-936171c0053e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f4157105f3488f807b7475008ab722', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:53.052141 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b6194f-150b-4616-aae5-936171c0053e service neutron] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:53.052515 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b6194f-150b-4616-aae5-936171c0053e service neutron] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:53.052756 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b6194f-150b-4616-aae5-936171c0053e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:53.069262 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b6194f-150b-4616-aae5-936171c0053e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:53.069262 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b6194f-150b-4616-aae5-936171c0053e service neutron] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:53.083277 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 934/1857] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 15:59:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:59:53.084601 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b33e309-7aac-4852-9790-b2344632ed49 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:53.084931 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b33e309-7aac-4852-9790-b2344632ed49 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:53.085148 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b33e309-7aac-4852-9790-b2344632ed49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:53.110566 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-0b33e309-7aac-4852-9790-b2344632ed49 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:59:53.112167 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-0b33e309-7aac-4852-9790-b2344632ed49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:59:53.126844 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 935/1858] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 15:59:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:59:53.131817 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b33e309-7aac-4852-9790-b2344632ed49 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:53.141595 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 924/1859] 10.222.0.20 () {58 vars in 1018 bytes} [Tue Aug 30 15:59:52 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:59:53.162224 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e547cc4-0efc-41c7-96a1-6bed3a8a9e01 None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:53.163779 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-5e547cc4-0efc-41c7-96a1-6bed3a8a9e01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:59:53.166730 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b33e309-7aac-4852-9790-b2344632ed49 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:53.178776 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82229f75-efc9-4826-9937-94ece309840c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:53.179397 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82229f75-efc9-4826-9937-94ece309840c admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:53.179675 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82229f75-efc9-4826-9937-94ece309840c admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:53.179942 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82229f75-efc9-4826-9937-94ece309840c admin admin] PATH_INFO: `/v3/domains` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:53.180810 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82229f75-efc9-4826-9937-94ece309840c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:59:53.181116 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82229f75-efc9-4826-9937-94ece309840c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:53.182963 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82229f75-efc9-4826-9937-94ece309840c admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:53.185263 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e547cc4-0efc-41c7-96a1-6bed3a8a9e01 None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:53.196598 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f716a017-3881-47fc-b237-89456169fde0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a1e756ae9be4a7bac1e81acf07e8f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:53.197957 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f716a017-3881-47fc-b237-89456169fde0 service nova] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:53.198364 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f716a017-3881-47fc-b237-89456169fde0 service nova] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:53.199813 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 936/1860] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Aug 30 15:59:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:59:53.202987 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f716a017-3881-47fc-b237-89456169fde0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:53.204609 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82229f75-efc9-4826-9937-94ece309840c admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:53.216370 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f716a017-3881-47fc-b237-89456169fde0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:53.219213 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f716a017-3881-47fc-b237-89456169fde0 service nova] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:53.220580 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82229f75-efc9-4826-9937-94ece309840c admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:53.231201 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09d541a-c54c-4aac-8b5e-83b8bc093a20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:53.231808 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09d541a-c54c-4aac-8b5e-83b8bc093a20 admin admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:53.233559 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09d541a-c54c-4aac-8b5e-83b8bc093a20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:53.236583 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09d541a-c54c-4aac-8b5e-83b8bc093a20 admin admin] PATH_INFO: `/v3/projects` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:53.237380 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09d541a-c54c-4aac-8b5e-83b8bc093a20 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:53.240062 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09d541a-c54c-4aac-8b5e-83b8bc093a20 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:53.243234 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 925/1861] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 15:59:53 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 15:59:53.284219 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 937/1862] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Aug 30 15:59:53 2022] POST /identity/v3/projects => generated 373 bytes in 80 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 15:59:53.288588 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f716a017-3881-47fc-b237-89456169fde0 service nova] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:53.303792 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f716a017-3881-47fc-b237-89456169fde0 service nova] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:53.425769 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:59:53.434417 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c6217d-2ee5-46b9-a242-906667dc9bbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:53.442105 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c6217d-2ee5-46b9-a242-906667dc9bbe admin admin] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:53.442544 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c6217d-2ee5-46b9-a242-906667dc9bbe admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:53.442824 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c6217d-2ee5-46b9-a242-906667dc9bbe admin admin] PATH_INFO: `/v3/users` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:53.443495 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c6217d-2ee5-46b9-a242-906667dc9bbe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:53.445689 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c6217d-2ee5-46b9-a242-906667dc9bbe admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:53.492425 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 926/1863] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Aug 30 15:59:53 2022] POST /identity/v3/users => generated 287 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 15:59:53.493161 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09d541a-c54c-4aac-8b5e-83b8bc093a20 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:53.505441 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09d541a-c54c-4aac-8b5e-83b8bc093a20 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:53.511985 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfea0da-dbcb-455f-b709-516d1f603c4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:53.512949 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfea0da-dbcb-455f-b709-516d1f603c4a admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:53.513566 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfea0da-dbcb-455f-b709-516d1f603c4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:53.514249 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfea0da-dbcb-455f-b709-516d1f603c4a admin admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:53.515051 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bfea0da-dbcb-455f-b709-516d1f603c4a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:53.516977 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bfea0da-dbcb-455f-b709-516d1f603c4a admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:53.531976 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 938/1864] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 30 15:59:53 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 15:59:53.535240 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c6217d-2ee5-46b9-a242-906667dc9bbe admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:53.546525 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c6217d-2ee5-46b9-a242-906667dc9bbe admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:53.552206 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8acad8-cde5-4b81-9eb9-a1ba81666157 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:53.553201 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8acad8-cde5-4b81-9eb9-a1ba81666157 admin admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:53.554366 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8acad8-cde5-4b81-9eb9-a1ba81666157 admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:53.554807 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8acad8-cde5-4b81-9eb9-a1ba81666157 admin admin] PATH_INFO: `/v3/roles` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:53.556222 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8acad8-cde5-4b81-9eb9-a1ba81666157 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:53.557253 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8acad8-cde5-4b81-9eb9-a1ba81666157 admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:53.573169 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 927/1865] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 30 15:59:53 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 15:59:53.574872 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfea0da-dbcb-455f-b709-516d1f603c4a admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:53.593856 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfea0da-dbcb-455f-b709-516d1f603c4a admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:53.601762 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489f3f2f-198d-4f7c-8f6b-7142ebc1eb4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:53.602830 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489f3f2f-198d-4f7c-8f6b-7142ebc1eb4a admin admin] REQUEST_METHOD: `PUT` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:53.603293 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489f3f2f-198d-4f7c-8f6b-7142ebc1eb4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:53.603742 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489f3f2f-198d-4f7c-8f6b-7142ebc1eb4a admin admin] PATH_INFO: `/v3/projects/c690bc11ee3d4565b7f63b8b65062242/users/bd289d086a9d44b4b816fd6362b33564/roles/df8f9f844b1b439181d75a07c1a93418` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:53.620670 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489f3f2f-198d-4f7c-8f6b-7142ebc1eb4a admin admin] RBAC: Authorizing `identity:create_grant(project_id=c690bc11ee3d4565b7f63b8b65062242, user_id=bd289d086a9d44b4b816fd6362b33564, role_id=df8f9f844b1b439181d75a07c1a93418)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:53.623392 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489f3f2f-198d-4f7c-8f6b-7142ebc1eb4a admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:53.644594 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 939/1866] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 30 15:59:53 2022] PUT /identity/v3/projects/c690bc11ee3d4565b7f63b8b65062242/users/bd289d086a9d44b4b816fd6362b33564/roles/df8f9f844b1b439181d75a07c1a93418 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:59:53.654945 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e04f4b-f136-490b-9881-33f6b24e7f84 None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:53.655510 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e04f4b-f136-490b-9881-33f6b24e7f84 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:53.655510 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e04f4b-f136-490b-9881-33f6b24e7f84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:53.691077 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-30e04f4b-f136-490b-9881-33f6b24e7f84 None None] MFA Rules not processed for user `bd289d086a9d44b4b816fd6362b33564`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:59:53.692451 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-30e04f4b-f136-490b-9881-33f6b24e7f84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:59:53.724902 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489f3f2f-198d-4f7c-8f6b-7142ebc1eb4a admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:53.726913 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-489f3f2f-198d-4f7c-8f6b-7142ebc1eb4a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:59:53.747894 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:59:53.872977 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489f3f2f-198d-4f7c-8f6b-7142ebc1eb4a admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:53.878776 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8271ad7c-2ccf-4d72-96de-b27187ce2ac5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:53.882945 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 928/1867] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 15:59:53 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:59:53.883155 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8271ad7c-2ccf-4d72-96de-b27187ce2ac5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:53.883155 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8271ad7c-2ccf-4d72-96de-b27187ce2ac5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:53.883155 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8271ad7c-2ccf-4d72-96de-b27187ce2ac5 admin admin] PATH_INFO: `/v3/projects/f0a12db0c8d94655998f379f0b389312` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:53.884059 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e04f4b-f136-490b-9881-33f6b24e7f84 None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:53.885155 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8271ad7c-2ccf-4d72-96de-b27187ce2ac5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f0a12db0c8d94655998f379f0b389312)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:53.887455 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8271ad7c-2ccf-4d72-96de-b27187ce2ac5 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:53.903938 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e04f4b-f136-490b-9881-33f6b24e7f84 None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:53.909972 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d736d2-0909-4c04-a7f6-2d697ebcd79e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:53.910733 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d736d2-0909-4c04-a7f6-2d697ebcd79e admin admin] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:53.911107 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d736d2-0909-4c04-a7f6-2d697ebcd79e admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:53.911474 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d736d2-0909-4c04-a7f6-2d697ebcd79e admin admin] PATH_INFO: `/v3/projects` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:53.912209 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d736d2-0909-4c04-a7f6-2d697ebcd79e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:53.915274 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d736d2-0909-4c04-a7f6-2d697ebcd79e admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:53.970318 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-8271ad7c-2ccf-4d72-96de-b27187ce2ac5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f0a12db0c8d94655998f379f0b389312', 'request_id': 'req-8271ad7c-2ccf-4d72-96de-b27187ce2ac5'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:59:53.983208 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-8271ad7c-2ccf-4d72-96de-b27187ce2ac5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f0a12db0c8d94655998f379f0b389312', 'request_id': 'req-8271ad7c-2ccf-4d72-96de-b27187ce2ac5'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:59:53.997456 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-8271ad7c-2ccf-4d72-96de-b27187ce2ac5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f0a12db0c8d94655998f379f0b389312', 'request_id': 'req-8271ad7c-2ccf-4d72-96de-b27187ce2ac5'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:59:54.017408 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 929/1868] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Aug 30 15:59:53 2022] POST /identity/v3/projects => generated 375 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 15:59:54.021338 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d736d2-0909-4c04-a7f6-2d697ebcd79e admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:54.035880 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-8271ad7c-2ccf-4d72-96de-b27187ce2ac5 admin admin] The token cache is being invalidate because project f0a12db0c8d94655998f379f0b389312 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81511) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 15:59:54.036168 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-8271ad7c-2ccf-4d72-96de-b27187ce2ac5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:59:54.039838 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-8271ad7c-2ccf-4d72-96de-b27187ce2ac5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:59:54.045673 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 940/1869] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Aug 30 15:59:53 2022] DELETE /identity/v3/projects/f0a12db0c8d94655998f379f0b389312 => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 15:59:54.049583 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8271ad7c-2ccf-4d72-96de-b27187ce2ac5 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:54.051461 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-8271ad7c-2ccf-4d72-96de-b27187ce2ac5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:59:54.126205 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d736d2-0909-4c04-a7f6-2d697ebcd79e admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:54.150205 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f22d4f-8f9d-4f89-babd-493a01415ee6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:54.154088 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f22d4f-8f9d-4f89-babd-493a01415ee6 admin admin] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:54.155271 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f22d4f-8f9d-4f89-babd-493a01415ee6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:54.155953 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f22d4f-8f9d-4f89-babd-493a01415ee6 admin admin] PATH_INFO: `/v3/users` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:54.160355 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f22d4f-8f9d-4f89-babd-493a01415ee6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:54.169279 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f22d4f-8f9d-4f89-babd-493a01415ee6 admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:54.198856 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8271ad7c-2ccf-4d72-96de-b27187ce2ac5 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:54.211203 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1b8449-e5a1-4c78-bf82-703a2a64d61e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:54.212120 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1b8449-e5a1-4c78-bf82-703a2a64d61e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:54.212730 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1b8449-e5a1-4c78-bf82-703a2a64d61e admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:54.213257 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1b8449-e5a1-4c78-bf82-703a2a64d61e admin admin] PATH_INFO: `/v3/users/3f04b5b3ace24840b276662f78048dd8` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:54.215181 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1b8449-e5a1-4c78-bf82-703a2a64d61e admin admin] RBAC: Authorizing `identity:delete_user(user_id=3f04b5b3ace24840b276662f78048dd8)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:54.217596 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1b8449-e5a1-4c78-bf82-703a2a64d61e admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:54.228599 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 930/1870] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Aug 30 15:59:54 2022] POST /identity/v3/users => generated 294 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 15:59:54.232288 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f22d4f-8f9d-4f89-babd-493a01415ee6 admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:54.234820 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-12f22d4f-8f9d-4f89-babd-493a01415ee6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:59:54.252614 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f22d4f-8f9d-4f89-babd-493a01415ee6 admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:54.260586 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42232bfb-85b8-4537-8a29-6b46be77ba0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:54.261361 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42232bfb-85b8-4537-8a29-6b46be77ba0e admin admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:54.261577 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42232bfb-85b8-4537-8a29-6b46be77ba0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:54.261853 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42232bfb-85b8-4537-8a29-6b46be77ba0e admin admin] PATH_INFO: `/v3/roles` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:54.262879 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42232bfb-85b8-4537-8a29-6b46be77ba0e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:54.264122 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42232bfb-85b8-4537-8a29-6b46be77ba0e admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:54.276429 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 931/1871] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 30 15:59:54 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 15:59:54.280783 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42232bfb-85b8-4537-8a29-6b46be77ba0e admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:54.297644 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42232bfb-85b8-4537-8a29-6b46be77ba0e admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:54.301866 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:59:54.305356 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a54eb0c-be52-405f-82e3-810d1d04a94c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:54.306407 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a54eb0c-be52-405f-82e3-810d1d04a94c admin admin] REQUEST_METHOD: `PUT` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:54.306790 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a54eb0c-be52-405f-82e3-810d1d04a94c admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:54.307165 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a54eb0c-be52-405f-82e3-810d1d04a94c admin admin] PATH_INFO: `/v3/projects/42bbdf53185b4fdbb23b72974b35d9ff/users/e625ec0194fb43a9b1ae859decec8bc8/roles/f265386950a641ac8413b53eda4ae34d` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:54.322743 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-fa1b8449-e5a1-4c78-bf82-703a2a64d61e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3f04b5b3ace24840b276662f78048dd8'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:59:54.333850 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a54eb0c-be52-405f-82e3-810d1d04a94c admin admin] RBAC: Authorizing `identity:create_grant(project_id=42bbdf53185b4fdbb23b72974b35d9ff, user_id=e625ec0194fb43a9b1ae859decec8bc8, role_id=f265386950a641ac8413b53eda4ae34d)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:54.340212 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a54eb0c-be52-405f-82e3-810d1d04a94c admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:54.343890 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:59:54.345340 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-fa1b8449-e5a1-4c78-bf82-703a2a64d61e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3f04b5b3ace24840b276662f78048dd8'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:59:54.369996 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-fa1b8449-e5a1-4c78-bf82-703a2a64d61e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3f04b5b3ace24840b276662f78048dd8'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:59:54.371265 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-fa1b8449-e5a1-4c78-bf82-703a2a64d61e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3f04b5b3ace24840b276662f78048dd8'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:59:54.376868 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 932/1872] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 30 15:59:54 2022] PUT /identity/v3/projects/42bbdf53185b4fdbb23b72974b35d9ff/users/e625ec0194fb43a9b1ae859decec8bc8/roles/f265386950a641ac8413b53eda4ae34d => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:59:54.379025 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a54eb0c-be52-405f-82e3-810d1d04a94c admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:54.380743 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-3a54eb0c-be52-405f-82e3-810d1d04a94c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:59:54.393125 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-fa1b8449-e5a1-4c78-bf82-703a2a64d61e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3f04b5b3ace24840b276662f78048dd8'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:59:54.408743 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 941/1873] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Aug 30 15:59:54 2022] DELETE /identity/v3/users/3f04b5b3ace24840b276662f78048dd8 => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:59:54.508832 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a54eb0c-be52-405f-82e3-810d1d04a94c admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:54.620807 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea2e830-72ef-423d-b587-39864eb0d518 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:54.621472 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea2e830-72ef-423d-b587-39864eb0d518 admin admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:54.621845 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea2e830-72ef-423d-b587-39864eb0d518 admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:54.622125 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea2e830-72ef-423d-b587-39864eb0d518 admin admin] PATH_INFO: `/v3/roles` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:54.622786 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea2e830-72ef-423d-b587-39864eb0d518 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:54.624732 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea2e830-72ef-423d-b587-39864eb0d518 admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:54.651327 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 933/1874] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 30 15:59:54 2022] GET /identity/v3/roles => generated 1420 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 15:59:54.655427 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1b8449-e5a1-4c78-bf82-703a2a64d61e admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:54.668977 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1b8449-e5a1-4c78-bf82-703a2a64d61e admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:54.673714 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b522d9f0-b037-4f3c-80f4-19057424ba3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:54.674449 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b522d9f0-b037-4f3c-80f4-19057424ba3b admin admin] REQUEST_METHOD: `PUT` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:54.674736 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b522d9f0-b037-4f3c-80f4-19057424ba3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:54.675018 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b522d9f0-b037-4f3c-80f4-19057424ba3b admin admin] PATH_INFO: `/v3/projects/42bbdf53185b4fdbb23b72974b35d9ff/users/e625ec0194fb43a9b1ae859decec8bc8/roles/f265386950a641ac8413b53eda4ae34d` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:54.677705 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b522d9f0-b037-4f3c-80f4-19057424ba3b admin admin] RBAC: Authorizing `identity:create_grant(project_id=42bbdf53185b4fdbb23b72974b35d9ff, user_id=e625ec0194fb43a9b1ae859decec8bc8, role_id=f265386950a641ac8413b53eda4ae34d)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:54.680966 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b522d9f0-b037-4f3c-80f4-19057424ba3b admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:54.706756 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 942/1875] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 30 15:59:54 2022] PUT /identity/v3/projects/42bbdf53185b4fdbb23b72974b35d9ff/users/e625ec0194fb43a9b1ae859decec8bc8/roles/f265386950a641ac8413b53eda4ae34d => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:59:54.707680 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc79942-a076-4240-bcd8-1df76a5fd1fb None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:54.708207 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc79942-a076-4240-bcd8-1df76a5fd1fb None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:54.708349 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc79942-a076-4240-bcd8-1df76a5fd1fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:54.749500 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-2dc79942-a076-4240-bcd8-1df76a5fd1fb None None] MFA Rules not processed for user `e625ec0194fb43a9b1ae859decec8bc8`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:59:54.750752 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-2dc79942-a076-4240-bcd8-1df76a5fd1fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:59:54.880871 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 934/1876] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 15:59:54 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:59:54.889419 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b522d9f0-b037-4f3c-80f4-19057424ba3b admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:54.891540 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-b522d9f0-b037-4f3c-80f4-19057424ba3b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:59:54.994311 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b522d9f0-b037-4f3c-80f4-19057424ba3b admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:54.998494 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03917473-8136-4d83-ae6f-ec1da8e32c36 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0100a04821804993866ea20f159663f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:54.999368 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03917473-8136-4d83-ae6f-ec1da8e32c36 service cinder] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:54.999490 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03917473-8136-4d83-ae6f-ec1da8e32c36 service cinder] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:54.999857 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03917473-8136-4d83-ae6f-ec1da8e32c36 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:55.002262 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-03917473-8136-4d83-ae6f-ec1da8e32c36 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:59:55.156840 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03917473-8136-4d83-ae6f-ec1da8e32c36 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:55.158673 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03917473-8136-4d83-ae6f-ec1da8e32c36 service cinder] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:55.171196 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 943/1877] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 15:59:54 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 282 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:59:55.373436 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc79942-a076-4240-bcd8-1df76a5fd1fb None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:55.375079 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-2dc79942-a076-4240-bcd8-1df76a5fd1fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:59:55.471386 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:59:55.524989 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc79942-a076-4240-bcd8-1df76a5fd1fb None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:55.531353 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57439695-2637-420a-9c6d-24159c0f7474 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:55.532002 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57439695-2637-420a-9c6d-24159c0f7474 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:55.532445 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57439695-2637-420a-9c6d-24159c0f7474 admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:55.532736 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57439695-2637-420a-9c6d-24159c0f7474 admin admin] PATH_INFO: `/v3/projects/db236552acb040b0b97956b3b41f84d0` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:55.534039 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57439695-2637-420a-9c6d-24159c0f7474 admin admin] RBAC: Authorizing `identity:delete_project(project_id=db236552acb040b0b97956b3b41f84d0)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:55.535995 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57439695-2637-420a-9c6d-24159c0f7474 admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:55.590060 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-57439695-2637-420a-9c6d-24159c0f7474 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'db236552acb040b0b97956b3b41f84d0', 'request_id': 'req-57439695-2637-420a-9c6d-24159c0f7474'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:59:55.602797 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-57439695-2637-420a-9c6d-24159c0f7474 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'db236552acb040b0b97956b3b41f84d0', 'request_id': 'req-57439695-2637-420a-9c6d-24159c0f7474'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:59:55.615191 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-57439695-2637-420a-9c6d-24159c0f7474 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'db236552acb040b0b97956b3b41f84d0', 'request_id': 'req-57439695-2637-420a-9c6d-24159c0f7474'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:59:55.629735 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-57439695-2637-420a-9c6d-24159c0f7474 admin admin] The token cache is being invalidate because project db236552acb040b0b97956b3b41f84d0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81512) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 15:59:55.630029 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-57439695-2637-420a-9c6d-24159c0f7474 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:59:55.630823 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-57439695-2637-420a-9c6d-24159c0f7474 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:59:55.632403 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 935/1878] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Aug 30 15:59:55 2022] DELETE /identity/v3/projects/db236552acb040b0b97956b3b41f84d0 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:59:55.637841 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2081b67e-a1a2-4008-a3f8-5c0588c6cd35 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:55.638222 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2081b67e-a1a2-4008-a3f8-5c0588c6cd35 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:55.638561 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2081b67e-a1a2-4008-a3f8-5c0588c6cd35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:55.664887 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-2081b67e-a1a2-4008-a3f8-5c0588c6cd35 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:59:55.665688 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-2081b67e-a1a2-4008-a3f8-5c0588c6cd35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:59:55.715283 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:59:55.728854 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 944/1879] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 15:59:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:59:55.737857 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7264c1-90ad-4a45-abcf-26d077dfde51 None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:55.738423 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7264c1-90ad-4a45-abcf-26d077dfde51 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:55.738423 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7264c1-90ad-4a45-abcf-26d077dfde51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:55.759069 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-ac7264c1-90ad-4a45-abcf-26d077dfde51 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:59:55.760217 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-ac7264c1-90ad-4a45-abcf-26d077dfde51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:59:55.768239 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 936/1880] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 15:59:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:59:55.772616 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2081b67e-a1a2-4008-a3f8-5c0588c6cd35 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:55.782276 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2081b67e-a1a2-4008-a3f8-5c0588c6cd35 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:55.787178 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db58c74-6191-41a1-af63-380833c966bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:55.787796 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db58c74-6191-41a1-af63-380833c966bc admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:55.788075 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db58c74-6191-41a1-af63-380833c966bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:55.788343 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db58c74-6191-41a1-af63-380833c966bc admin admin] PATH_INFO: `/v3/domains` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:55.788992 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db58c74-6191-41a1-af63-380833c966bc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:59:55.789324 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db58c74-6191-41a1-af63-380833c966bc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:55.790978 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db58c74-6191-41a1-af63-380833c966bc admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:55.803460 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 945/1881] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Aug 30 15:59:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:59:55.808010 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7264c1-90ad-4a45-abcf-26d077dfde51 None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:55.817831 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7264c1-90ad-4a45-abcf-26d077dfde51 None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:55.822964 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b454f689-ef6f-4cf1-964c-1144c41fb47a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:55.823596 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b454f689-ef6f-4cf1-964c-1144c41fb47a admin admin] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:55.823918 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b454f689-ef6f-4cf1-964c-1144c41fb47a admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:55.824193 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b454f689-ef6f-4cf1-964c-1144c41fb47a admin admin] PATH_INFO: `/v3/projects` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:55.824872 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b454f689-ef6f-4cf1-964c-1144c41fb47a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:55.826881 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b454f689-ef6f-4cf1-964c-1144c41fb47a admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:55.866246 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 937/1882] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Aug 30 15:59:55 2022] POST /identity/v3/projects => generated 387 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 15:59:55.867591 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db58c74-6191-41a1-af63-380833c966bc admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:55.878447 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db58c74-6191-41a1-af63-380833c966bc admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:55.953974 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750b314a-2f43-48f1-9429-dce89df985fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:55.954607 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750b314a-2f43-48f1-9429-dce89df985fe admin admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:55.954881 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750b314a-2f43-48f1-9429-dce89df985fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:55.955150 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750b314a-2f43-48f1-9429-dce89df985fe admin admin] PATH_INFO: `/v3/users` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:55.955788 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750b314a-2f43-48f1-9429-dce89df985fe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:55.957756 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750b314a-2f43-48f1-9429-dce89df985fe admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:55.999432 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 946/1883] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Aug 30 15:59:55 2022] POST /identity/v3/users => generated 294 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 15:59:56.003923 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b454f689-ef6f-4cf1-964c-1144c41fb47a admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:56.021469 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b454f689-ef6f-4cf1-964c-1144c41fb47a admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:56.027405 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0b9620-d1da-4c91-bded-1336dc428965 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:56.028028 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0b9620-d1da-4c91-bded-1336dc428965 admin admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:56.028308 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0b9620-d1da-4c91-bded-1336dc428965 admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:56.028579 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0b9620-d1da-4c91-bded-1336dc428965 admin admin] PATH_INFO: `/v3/roles` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:56.029335 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0b9620-d1da-4c91-bded-1336dc428965 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:56.031786 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0b9620-d1da-4c91-bded-1336dc428965 admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:56.054419 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 938/1884] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 30 15:59:56 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 15:59:56.055087 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750b314a-2f43-48f1-9429-dce89df985fe admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:56.074098 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750b314a-2f43-48f1-9429-dce89df985fe admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:56.082607 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a20197d-2358-4649-908b-22066ccb1ce2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:56.083335 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a20197d-2358-4649-908b-22066ccb1ce2 admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:56.083891 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a20197d-2358-4649-908b-22066ccb1ce2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:56.084487 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a20197d-2358-4649-908b-22066ccb1ce2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:56.085192 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a20197d-2358-4649-908b-22066ccb1ce2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:56.103458 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a20197d-2358-4649-908b-22066ccb1ce2 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:56.176582 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0b9620-d1da-4c91-bded-1336dc428965 admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:56.182248 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 947/1885] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 30 15:59:56 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 15:59:56.189240 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0b9620-d1da-4c91-bded-1336dc428965 admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:56.201987 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed04d9e5-74bc-47fc-b71b-ecd7d3de1d66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:56.203363 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed04d9e5-74bc-47fc-b71b-ecd7d3de1d66 admin admin] REQUEST_METHOD: `PUT` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:56.203911 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed04d9e5-74bc-47fc-b71b-ecd7d3de1d66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:56.204554 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed04d9e5-74bc-47fc-b71b-ecd7d3de1d66 admin admin] PATH_INFO: `/v3/projects/8bcd895c564f468b9274fe9a06871a95/users/10500d514df04c328e0f36e85ba740cd/roles/df8f9f844b1b439181d75a07c1a93418` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:56.224863 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed04d9e5-74bc-47fc-b71b-ecd7d3de1d66 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8bcd895c564f468b9274fe9a06871a95, user_id=10500d514df04c328e0f36e85ba740cd, role_id=df8f9f844b1b439181d75a07c1a93418)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:56.229139 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed04d9e5-74bc-47fc-b71b-ecd7d3de1d66 admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:56.248779 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 939/1886] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 30 15:59:56 2022] PUT /identity/v3/projects/8bcd895c564f468b9274fe9a06871a95/users/10500d514df04c328e0f36e85ba740cd/roles/df8f9f844b1b439181d75a07c1a93418 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:59:56.259132 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81aaae1-4b8a-4087-b97b-815c3ac21c18 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:56.259484 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81aaae1-4b8a-4087-b97b-815c3ac21c18 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:56.259681 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81aaae1-4b8a-4087-b97b-815c3ac21c18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:56.348360 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-c81aaae1-4b8a-4087-b97b-815c3ac21c18 None None] MFA Rules not processed for user `10500d514df04c328e0f36e85ba740cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:59:56.354796 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-c81aaae1-4b8a-4087-b97b-815c3ac21c18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:59:56.562956 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 948/1887] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 15:59:56 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 305 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:59:56.570125 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed04d9e5-74bc-47fc-b71b-ecd7d3de1d66 admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:56.658802 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:59:56.675640 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed04d9e5-74bc-47fc-b71b-ecd7d3de1d66 admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:56.684558 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f5419e-87a5-4fbb-91d1-14b822f87191 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:56.685505 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f5419e-87a5-4fbb-91d1-14b822f87191 admin admin] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:56.686113 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f5419e-87a5-4fbb-91d1-14b822f87191 admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:56.686802 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f5419e-87a5-4fbb-91d1-14b822f87191 admin admin] PATH_INFO: `/v3/projects` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:56.687745 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f5419e-87a5-4fbb-91d1-14b822f87191 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:56.689827 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f5419e-87a5-4fbb-91d1-14b822f87191 admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:56.769768 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 940/1888] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Aug 30 15:59:56 2022] POST /identity/v3/projects => generated 387 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 15:59:56.775353 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81aaae1-4b8a-4087-b97b-815c3ac21c18 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:56.814777 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81aaae1-4b8a-4087-b97b-815c3ac21c18 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:56.903020 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:59:56.924671 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9def5b8-c5d4-4d7e-a7a9-2d10f21cfa11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:56.926531 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9def5b8-c5d4-4d7e-a7a9-2d10f21cfa11 admin admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:56.926531 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9def5b8-c5d4-4d7e-a7a9-2d10f21cfa11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:56.926531 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9def5b8-c5d4-4d7e-a7a9-2d10f21cfa11 admin admin] PATH_INFO: `/v3/users` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:56.927134 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9def5b8-c5d4-4d7e-a7a9-2d10f21cfa11 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:56.929182 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9def5b8-c5d4-4d7e-a7a9-2d10f21cfa11 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:56.957463 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 949/1889] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Aug 30 15:59:56 2022] POST /identity/v3/users => generated 300 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 15:59:56.961859 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f5419e-87a5-4fbb-91d1-14b822f87191 admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:56.972580 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f5419e-87a5-4fbb-91d1-14b822f87191 admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:56.978319 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fc046b-435c-44fd-adf4-29e9b4682387 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:56.979158 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fc046b-435c-44fd-adf4-29e9b4682387 admin admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:56.979594 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fc046b-435c-44fd-adf4-29e9b4682387 admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:56.980048 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fc046b-435c-44fd-adf4-29e9b4682387 admin admin] PATH_INFO: `/v3/roles` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:56.980860 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fc046b-435c-44fd-adf4-29e9b4682387 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:56.982723 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fc046b-435c-44fd-adf4-29e9b4682387 admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:56.997205 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 941/1890] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 30 15:59:56 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 15:59:56.998670 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9def5b8-c5d4-4d7e-a7a9-2d10f21cfa11 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:57.009366 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9def5b8-c5d4-4d7e-a7a9-2d10f21cfa11 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:57.014322 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1c481e-6327-47c3-bbbf-222691ad096e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:57.015098 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1c481e-6327-47c3-bbbf-222691ad096e admin admin] REQUEST_METHOD: `PUT` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:57.015466 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1c481e-6327-47c3-bbbf-222691ad096e admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:57.015954 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1c481e-6327-47c3-bbbf-222691ad096e admin admin] PATH_INFO: `/v3/projects/c1ba5c61edf9465580593a84cab97e9a/users/d4ec0b0c4e4c47339a5f1d0b1de9b2e3/roles/f265386950a641ac8413b53eda4ae34d` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:57.032283 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1c481e-6327-47c3-bbbf-222691ad096e admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1ba5c61edf9465580593a84cab97e9a, user_id=d4ec0b0c4e4c47339a5f1d0b1de9b2e3, role_id=f265386950a641ac8413b53eda4ae34d)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:57.035018 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1c481e-6327-47c3-bbbf-222691ad096e admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:57.050444 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 950/1891] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 30 15:59:56 2022] PUT /identity/v3/projects/c1ba5c61edf9465580593a84cab97e9a/users/d4ec0b0c4e4c47339a5f1d0b1de9b2e3/roles/f265386950a641ac8413b53eda4ae34d => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:59:57.054421 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fc046b-435c-44fd-adf4-29e9b4682387 admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:57.065990 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fc046b-435c-44fd-adf4-29e9b4682387 admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:57.126014 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805d5b9b-af3f-4110-ba12-10a75a692a79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:57.126657 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805d5b9b-af3f-4110-ba12-10a75a692a79 admin admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:57.126930 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805d5b9b-af3f-4110-ba12-10a75a692a79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:57.127196 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805d5b9b-af3f-4110-ba12-10a75a692a79 admin admin] PATH_INFO: `/v3/roles` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:57.127876 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805d5b9b-af3f-4110-ba12-10a75a692a79 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:57.129504 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805d5b9b-af3f-4110-ba12-10a75a692a79 admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:57.139947 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 942/1892] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 30 15:59:57 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 15:59:57.144276 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1c481e-6327-47c3-bbbf-222691ad096e admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:57.154821 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1c481e-6327-47c3-bbbf-222691ad096e admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:57.159525 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f52f066-dc58-4473-80c8-b9ba6671b4ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:57.160217 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f52f066-dc58-4473-80c8-b9ba6671b4ef admin admin] REQUEST_METHOD: `PUT` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:57.160498 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f52f066-dc58-4473-80c8-b9ba6671b4ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:57.160767 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f52f066-dc58-4473-80c8-b9ba6671b4ef admin admin] PATH_INFO: `/v3/projects/c1ba5c61edf9465580593a84cab97e9a/users/d4ec0b0c4e4c47339a5f1d0b1de9b2e3/roles/f265386950a641ac8413b53eda4ae34d` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:57.162806 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f52f066-dc58-4473-80c8-b9ba6671b4ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1ba5c61edf9465580593a84cab97e9a, user_id=d4ec0b0c4e4c47339a5f1d0b1de9b2e3, role_id=f265386950a641ac8413b53eda4ae34d)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:57.165531 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f52f066-dc58-4473-80c8-b9ba6671b4ef admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:57.176221 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 951/1893] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 30 15:59:57 2022] PUT /identity/v3/projects/c1ba5c61edf9465580593a84cab97e9a/users/d4ec0b0c4e4c47339a5f1d0b1de9b2e3/roles/f265386950a641ac8413b53eda4ae34d => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:59:57.186842 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f25886-c627-4144-aa7b-7bb7b8af70e9 None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:57.187114 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f25886-c627-4144-aa7b-7bb7b8af70e9 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:57.187433 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f25886-c627-4144-aa7b-7bb7b8af70e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:57.225930 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-b0f25886-c627-4144-aa7b-7bb7b8af70e9 None None] MFA Rules not processed for user `d4ec0b0c4e4c47339a5f1d0b1de9b2e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:59:57.227280 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-b0f25886-c627-4144-aa7b-7bb7b8af70e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:59:57.319340 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 943/1894] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 15:59:57 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:59:57.330420 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f52f066-dc58-4473-80c8-b9ba6671b4ef admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:57.332360 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-2f52f066-dc58-4473-80c8-b9ba6671b4ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:59:57.413125 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f52f066-dc58-4473-80c8-b9ba6671b4ef admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:57.417611 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e4e319-b0c2-4a91-8246-7a0403b2360e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0100a04821804993866ea20f159663f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:57.418518 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e4e319-b0c2-4a91-8246-7a0403b2360e service cinder] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:57.418913 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e4e319-b0c2-4a91-8246-7a0403b2360e service cinder] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:57.419319 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e4e319-b0c2-4a91-8246-7a0403b2360e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:57.433453 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e4e319-b0c2-4a91-8246-7a0403b2360e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:57.435166 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e4e319-b0c2-4a91-8246-7a0403b2360e service cinder] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:57.446106 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 952/1895] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 15:59:57 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:59:57.522112 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f25886-c627-4144-aa7b-7bb7b8af70e9 None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:57.569933 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:59:57.588608 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f25886-c627-4144-aa7b-7bb7b8af70e9 None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:57.594443 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2f171b-8b03-453f-aeab-9e85b806bc83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:57.595096 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2f171b-8b03-453f-aeab-9e85b806bc83 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:57.595370 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2f171b-8b03-453f-aeab-9e85b806bc83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:57.595660 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2f171b-8b03-453f-aeab-9e85b806bc83 admin admin] PATH_INFO: `/v3/users/10500d514df04c328e0f36e85ba740cd` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:57.597047 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2f171b-8b03-453f-aeab-9e85b806bc83 admin admin] RBAC: Authorizing `identity:delete_user(user_id=10500d514df04c328e0f36e85ba740cd)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:57.599262 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2f171b-8b03-453f-aeab-9e85b806bc83 admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:57.662362 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-4c2f171b-8b03-453f-aeab-9e85b806bc83 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '10500d514df04c328e0f36e85ba740cd'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:59:57.673462 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-4c2f171b-8b03-453f-aeab-9e85b806bc83 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '10500d514df04c328e0f36e85ba740cd'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:59:57.700780 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-4c2f171b-8b03-453f-aeab-9e85b806bc83 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '10500d514df04c328e0f36e85ba740cd'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:59:57.709458 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-4c2f171b-8b03-453f-aeab-9e85b806bc83 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '10500d514df04c328e0f36e85ba740cd'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:59:57.710272 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-4c2f171b-8b03-453f-aeab-9e85b806bc83 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '10500d514df04c328e0f36e85ba740cd'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:59:57.719033 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 944/1896] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Aug 30 15:59:57 2022] DELETE /identity/v3/users/10500d514df04c328e0f36e85ba740cd => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:59:57.723443 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f25ed28-4bf9-4f06-88a2-9b4d1d367933 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:57.723730 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f25ed28-4bf9-4f06-88a2-9b4d1d367933 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:57.724001 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f25ed28-4bf9-4f06-88a2-9b4d1d367933 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:57.746014 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-8f25ed28-4bf9-4f06-88a2-9b4d1d367933 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:59:57.747922 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-8f25ed28-4bf9-4f06-88a2-9b4d1d367933 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:59:57.833247 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 953/1897] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 15:59:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:59:57.844161 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2f171b-8b03-453f-aeab-9e85b806bc83 admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:57.848162 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-4c2f171b-8b03-453f-aeab-9e85b806bc83 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:59:57.935656 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2f171b-8b03-453f-aeab-9e85b806bc83 admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:57.940134 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2111e9b-d481-4a92-b207-e9d7ae5fe8a6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f4157105f3488f807b7475008ab722', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:57.940960 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2111e9b-d481-4a92-b207-e9d7ae5fe8a6 service neutron] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:57.941261 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2111e9b-d481-4a92-b207-e9d7ae5fe8a6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:57.943686 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2111e9b-d481-4a92-b207-e9d7ae5fe8a6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:58.039853 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2111e9b-d481-4a92-b207-e9d7ae5fe8a6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:58.040891 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f25ed28-4bf9-4f06-88a2-9b4d1d367933 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:58.041753 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2111e9b-d481-4a92-b207-e9d7ae5fe8a6 service neutron] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:58.043099 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-8f25ed28-4bf9-4f06-88a2-9b4d1d367933 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:59:58.067622 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f25ed28-4bf9-4f06-88a2-9b4d1d367933 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:58.081635 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1720ff06-7a7b-4276-91e1-59ab1eebcd5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:58.081799 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1720ff06-7a7b-4276-91e1-59ab1eebcd5e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:58.082794 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1720ff06-7a7b-4276-91e1-59ab1eebcd5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:58.083090 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1720ff06-7a7b-4276-91e1-59ab1eebcd5e admin admin] PATH_INFO: `/v3/users/bd289d086a9d44b4b816fd6362b33564` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:58.086230 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1720ff06-7a7b-4276-91e1-59ab1eebcd5e admin admin] RBAC: Authorizing `identity:delete_user(user_id=bd289d086a9d44b4b816fd6362b33564)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:58.095222 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1720ff06-7a7b-4276-91e1-59ab1eebcd5e admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:58.097797 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 945/1898] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 15:59:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 254 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:59:58.195456 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=90,keystone:INSERT=5,keystone:DELETE=5 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:59:58.207016 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-1720ff06-7a7b-4276-91e1-59ab1eebcd5e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bd289d086a9d44b4b816fd6362b33564'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:59:58.216337 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c43029a-3527-41c9-a5d0-a5771672a6cb None None] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:58.216661 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c43029a-3527-41c9-a5d0-a5771672a6cb None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:58.216932 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c43029a-3527-41c9-a5d0-a5771672a6cb None None] PATH_INFO: `/` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:58.218632 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 946/1899] 10.222.0.20 () {54 vars in 921 bytes} [Tue Aug 30 15:59:58 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 15:59:58.235039 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c926710e-5762-47fd-a583-e6d2e1523ad9 None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:58.235327 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c926710e-5762-47fd-a583-e6d2e1523ad9 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:58.235646 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c926710e-5762-47fd-a583-e6d2e1523ad9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:58.249015 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-1720ff06-7a7b-4276-91e1-59ab1eebcd5e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bd289d086a9d44b4b816fd6362b33564'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:59:58.271130 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-1720ff06-7a7b-4276-91e1-59ab1eebcd5e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bd289d086a9d44b4b816fd6362b33564'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:59:58.272676 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-c926710e-5762-47fd-a583-e6d2e1523ad9 None None] MFA Rules not processed for user `7a1e756ae9be4a7bac1e81acf07e8f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:59:58.273782 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-1720ff06-7a7b-4276-91e1-59ab1eebcd5e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bd289d086a9d44b4b816fd6362b33564'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:59:58.275719 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-c926710e-5762-47fd-a583-e6d2e1523ad9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:59:58.299084 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-1720ff06-7a7b-4276-91e1-59ab1eebcd5e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bd289d086a9d44b4b816fd6362b33564'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:59:58.306788 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 954/1900] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Aug 30 15:59:58 2022] DELETE /identity/v3/users/bd289d086a9d44b4b816fd6362b33564 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 15:59:58.312075 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef79e598-dcd5-4a3e-8601-d2c2e238b28b None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:58.313430 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef79e598-dcd5-4a3e-8601-d2c2e238b28b None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:58.313902 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef79e598-dcd5-4a3e-8601-d2c2e238b28b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:58.359964 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-ef79e598-dcd5-4a3e-8601-d2c2e238b28b None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:59:58.362622 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-ef79e598-dcd5-4a3e-8601-d2c2e238b28b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:59:58.409061 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:59:58.422797 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 947/1901] 10.222.0.20 () {58 vars in 1018 bytes} [Tue Aug 30 15:59:58 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:59:58.456426 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 955/1902] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 15:59:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:59:58.468493 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c926710e-5762-47fd-a583-e6d2e1523ad9 None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:58.470188 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-c926710e-5762-47fd-a583-e6d2e1523ad9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:59:58.486219 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c926710e-5762-47fd-a583-e6d2e1523ad9 None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:58.578743 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93c1226-d1a1-45d3-8a63-c579b9a9088a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f4157105f3488f807b7475008ab722', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:58.579467 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93c1226-d1a1-45d3-8a63-c579b9a9088a service neutron] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:58.579752 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93c1226-d1a1-45d3-8a63-c579b9a9088a service neutron] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:58.580082 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93c1226-d1a1-45d3-8a63-c579b9a9088a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:58.582979 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef79e598-dcd5-4a3e-8601-d2c2e238b28b None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:58.584997 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-ef79e598-dcd5-4a3e-8601-d2c2e238b28b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:59:58.591358 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93c1226-d1a1-45d3-8a63-c579b9a9088a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:58.592988 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93c1226-d1a1-45d3-8a63-c579b9a9088a service neutron] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:58.612365 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 948/1903] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 15:59:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:59:58.617204 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93c1226-d1a1-45d3-8a63-c579b9a9088a service neutron] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:58.617204 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-f93c1226-d1a1-45d3-8a63-c579b9a9088a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:59:58.647367 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93c1226-d1a1-45d3-8a63-c579b9a9088a service neutron] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:58.653067 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a213f99b-f8ca-40a5-8f1f-b0356cac2cb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:58.653886 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a213f99b-f8ca-40a5-8f1f-b0356cac2cb0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:58.654182 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a213f99b-f8ca-40a5-8f1f-b0356cac2cb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:58.654429 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a213f99b-f8ca-40a5-8f1f-b0356cac2cb0 admin admin] PATH_INFO: `/v3/projects/8bcd895c564f468b9274fe9a06871a95` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:58.655631 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a213f99b-f8ca-40a5-8f1f-b0356cac2cb0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8bcd895c564f468b9274fe9a06871a95)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:58.657679 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a213f99b-f8ca-40a5-8f1f-b0356cac2cb0 admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:58.711394 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef79e598-dcd5-4a3e-8601-d2c2e238b28b None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:58.713968 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-a213f99b-f8ca-40a5-8f1f-b0356cac2cb0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8bcd895c564f468b9274fe9a06871a95', 'request_id': 'req-a213f99b-f8ca-40a5-8f1f-b0356cac2cb0'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:59:58.729013 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-a213f99b-f8ca-40a5-8f1f-b0356cac2cb0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8bcd895c564f468b9274fe9a06871a95', 'request_id': 'req-a213f99b-f8ca-40a5-8f1f-b0356cac2cb0'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:59:58.738863 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-a213f99b-f8ca-40a5-8f1f-b0356cac2cb0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8bcd895c564f468b9274fe9a06871a95', 'request_id': 'req-a213f99b-f8ca-40a5-8f1f-b0356cac2cb0'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:59:58.752390 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-a213f99b-f8ca-40a5-8f1f-b0356cac2cb0 admin admin] The token cache is being invalidate because project 8bcd895c564f468b9274fe9a06871a95 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81512) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 15:59:58.753016 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-a213f99b-f8ca-40a5-8f1f-b0356cac2cb0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:59:58.753720 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-a213f99b-f8ca-40a5-8f1f-b0356cac2cb0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:59:58.759744 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 949/1904] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Aug 30 15:59:58 2022] DELETE /identity/v3/projects/8bcd895c564f468b9274fe9a06871a95 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:59:58.760711 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a213f99b-f8ca-40a5-8f1f-b0356cac2cb0 admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:58.762570 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-a213f99b-f8ca-40a5-8f1f-b0356cac2cb0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:59:58.791071 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:59:58.831630 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f4985b-5771-456c-b43d-1e81c0b148a5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a1e756ae9be4a7bac1e81acf07e8f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:58.832299 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f4985b-5771-456c-b43d-1e81c0b148a5 service nova] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:58.833601 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f4985b-5771-456c-b43d-1e81c0b148a5 service nova] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:58.834588 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f4985b-5771-456c-b43d-1e81c0b148a5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:58.837607 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-13f4985b-5771-456c-b43d-1e81c0b148a5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:59:58.852264 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a213f99b-f8ca-40a5-8f1f-b0356cac2cb0 admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:58.853650 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f4985b-5771-456c-b43d-1e81c0b148a5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:58.855433 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f4985b-5771-456c-b43d-1e81c0b148a5 service nova] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:58.857652 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8525e44-5f94-420b-a4a2-9244b29064e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:58.858372 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8525e44-5f94-420b-a4a2-9244b29064e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:58.858724 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8525e44-5f94-420b-a4a2-9244b29064e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:58.859070 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8525e44-5f94-420b-a4a2-9244b29064e5 admin admin] PATH_INFO: `/v3/users/d4ec0b0c4e4c47339a5f1d0b1de9b2e3` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:58.860301 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8525e44-5f94-420b-a4a2-9244b29064e5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d4ec0b0c4e4c47339a5f1d0b1de9b2e3)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:58.862369 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8525e44-5f94-420b-a4a2-9244b29064e5 admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:58.866980 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 956/1905] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 15:59:58 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:59:58.896717 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:59:58.923149 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f4985b-5771-456c-b43d-1e81c0b148a5 service nova] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:58.924668 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-13f4985b-5771-456c-b43d-1e81c0b148a5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:59:58.927045 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-d8525e44-5f94-420b-a4a2-9244b29064e5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd4ec0b0c4e4c47339a5f1d0b1de9b2e3'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:59:58.937768 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-d8525e44-5f94-420b-a4a2-9244b29064e5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd4ec0b0c4e4c47339a5f1d0b1de9b2e3'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:59:58.939368 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f4985b-5771-456c-b43d-1e81c0b148a5 service nova] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:58.944160 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715a72fc-cad0-4bf3-b3d1-338a443f4962 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:58.944802 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715a72fc-cad0-4bf3-b3d1-338a443f4962 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:58.945204 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715a72fc-cad0-4bf3-b3d1-338a443f4962 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:58.945517 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715a72fc-cad0-4bf3-b3d1-338a443f4962 admin admin] PATH_INFO: `/v3/projects/c690bc11ee3d4565b7f63b8b65062242` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:58.947610 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715a72fc-cad0-4bf3-b3d1-338a443f4962 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c690bc11ee3d4565b7f63b8b65062242)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:58.949756 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715a72fc-cad0-4bf3-b3d1-338a443f4962 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:58.952418 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-d8525e44-5f94-420b-a4a2-9244b29064e5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd4ec0b0c4e4c47339a5f1d0b1de9b2e3'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:59:58.960185 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-d8525e44-5f94-420b-a4a2-9244b29064e5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd4ec0b0c4e4c47339a5f1d0b1de9b2e3'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:59:58.961271 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-d8525e44-5f94-420b-a4a2-9244b29064e5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd4ec0b0c4e4c47339a5f1d0b1de9b2e3'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:59:58.967369 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 950/1906] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Aug 30 15:59:58 2022] DELETE /identity/v3/users/d4ec0b0c4e4c47339a5f1d0b1de9b2e3 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:59:59.006066 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-715a72fc-cad0-4bf3-b3d1-338a443f4962 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c690bc11ee3d4565b7f63b8b65062242', 'request_id': 'req-715a72fc-cad0-4bf3-b3d1-338a443f4962'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:59:59.016154 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-715a72fc-cad0-4bf3-b3d1-338a443f4962 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c690bc11ee3d4565b7f63b8b65062242', 'request_id': 'req-715a72fc-cad0-4bf3-b3d1-338a443f4962'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:59:59.026125 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-715a72fc-cad0-4bf3-b3d1-338a443f4962 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c690bc11ee3d4565b7f63b8b65062242', 'request_id': 'req-715a72fc-cad0-4bf3-b3d1-338a443f4962'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:59:59.039426 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-715a72fc-cad0-4bf3-b3d1-338a443f4962 admin admin] The token cache is being invalidate because project c690bc11ee3d4565b7f63b8b65062242 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81511) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 15:59:59.039718 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-715a72fc-cad0-4bf3-b3d1-338a443f4962 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:59:59.040407 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-715a72fc-cad0-4bf3-b3d1-338a443f4962 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:59:59.042017 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 957/1907] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Aug 30 15:59:58 2022] DELETE /identity/v3/projects/c690bc11ee3d4565b7f63b8b65062242 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 15:59:59.046171 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8525e44-5f94-420b-a4a2-9244b29064e5 admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:59.047651 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-d8525e44-5f94-420b-a4a2-9244b29064e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:59:59.120260 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8525e44-5f94-420b-a4a2-9244b29064e5 admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:59:59.205032 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9984783c-1ad2-4494-960a-0e1c3e18f6ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:59:59.205744 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9984783c-1ad2-4494-960a-0e1c3e18f6ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:59:59.206029 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9984783c-1ad2-4494-960a-0e1c3e18f6ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:59:59.207682 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9984783c-1ad2-4494-960a-0e1c3e18f6ab admin admin] PATH_INFO: `/v3/users/e625ec0194fb43a9b1ae859decec8bc8` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:59:59.209144 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9984783c-1ad2-4494-960a-0e1c3e18f6ab admin admin] RBAC: Authorizing `identity:delete_user(user_id=e625ec0194fb43a9b1ae859decec8bc8)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:59:59.211388 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9984783c-1ad2-4494-960a-0e1c3e18f6ab admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:59:59.298173 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-9984783c-1ad2-4494-960a-0e1c3e18f6ab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e625ec0194fb43a9b1ae859decec8bc8'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:59:59.319727 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-9984783c-1ad2-4494-960a-0e1c3e18f6ab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e625ec0194fb43a9b1ae859decec8bc8'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:59:59.321758 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:59:59.341642 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-9984783c-1ad2-4494-960a-0e1c3e18f6ab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e625ec0194fb43a9b1ae859decec8bc8'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:59:59.358843 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-9984783c-1ad2-4494-960a-0e1c3e18f6ab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e625ec0194fb43a9b1ae859decec8bc8'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:59:59.359645 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-9984783c-1ad2-4494-960a-0e1c3e18f6ab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e625ec0194fb43a9b1ae859decec8bc8'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:59:59.371878 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 951/1908] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Aug 30 15:59:59 2022] DELETE /identity/v3/users/e625ec0194fb43a9b1ae859decec8bc8 => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 15:59:59.914991 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715a72fc-cad0-4bf3-b3d1-338a443f4962 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:59:59.916479 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-715a72fc-cad0-4bf3-b3d1-338a443f4962 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:59:59.938263 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715a72fc-cad0-4bf3-b3d1-338a443f4962 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:00.016285 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070eda9d-bc95-4e28-8620-5bd58f488569 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:00.017283 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070eda9d-bc95-4e28-8620-5bd58f488569 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:00.017688 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070eda9d-bc95-4e28-8620-5bd58f488569 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:00.018092 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070eda9d-bc95-4e28-8620-5bd58f488569 admin admin] PATH_INFO: `/v3/projects/42bbdf53185b4fdbb23b72974b35d9ff` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:00.019546 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070eda9d-bc95-4e28-8620-5bd58f488569 admin admin] RBAC: Authorizing `identity:delete_project(project_id=42bbdf53185b4fdbb23b72974b35d9ff)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:00.022232 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070eda9d-bc95-4e28-8620-5bd58f488569 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:00.131559 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-070eda9d-bc95-4e28-8620-5bd58f488569 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '42bbdf53185b4fdbb23b72974b35d9ff', 'request_id': 'req-070eda9d-bc95-4e28-8620-5bd58f488569'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:00.141064 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9984783c-1ad2-4494-960a-0e1c3e18f6ab admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:00.142745 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-9984783c-1ad2-4494-960a-0e1c3e18f6ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:00:00.144203 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-070eda9d-bc95-4e28-8620-5bd58f488569 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '42bbdf53185b4fdbb23b72974b35d9ff', 'request_id': 'req-070eda9d-bc95-4e28-8620-5bd58f488569'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:00.155851 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-070eda9d-bc95-4e28-8620-5bd58f488569 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '42bbdf53185b4fdbb23b72974b35d9ff', 'request_id': 'req-070eda9d-bc95-4e28-8620-5bd58f488569'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:00.161547 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:00:00.185106 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9984783c-1ad2-4494-960a-0e1c3e18f6ab admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:00.188374 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-070eda9d-bc95-4e28-8620-5bd58f488569 admin admin] The token cache is being invalidate because project 42bbdf53185b4fdbb23b72974b35d9ff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81511) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 16:00:00.188778 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-070eda9d-bc95-4e28-8620-5bd58f488569 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:00.189576 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-070eda9d-bc95-4e28-8620-5bd58f488569 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:00.196451 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 958/1909] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Aug 30 15:59:59 2022] DELETE /identity/v3/projects/42bbdf53185b4fdbb23b72974b35d9ff => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:00:00.200202 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5288462d-79d7-49c4-be39-7106d6985146 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:00.202091 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5288462d-79d7-49c4-be39-7106d6985146 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:00.203235 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5288462d-79d7-49c4-be39-7106d6985146 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:00.268231 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9165334f-9f8e-4b8c-9021-30eb3c9503b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:00.269009 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9165334f-9f8e-4b8c-9021-30eb3c9503b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:00.269331 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9165334f-9f8e-4b8c-9021-30eb3c9503b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:00.269575 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-5288462d-79d7-49c4-be39-7106d6985146 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:00:00.269696 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9165334f-9f8e-4b8c-9021-30eb3c9503b9 admin admin] PATH_INFO: `/v3/projects/c1ba5c61edf9465580593a84cab97e9a` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:00.270943 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9165334f-9f8e-4b8c-9021-30eb3c9503b9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c1ba5c61edf9465580593a84cab97e9a)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:00.271071 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-5288462d-79d7-49c4-be39-7106d6985146 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:00:00.273098 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9165334f-9f8e-4b8c-9021-30eb3c9503b9 admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:00.291139 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 959/1910] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 16:00:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:00:00.299879 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b999dd17-753b-47d6-a7c9-7490cec38297 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:00.300162 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b999dd17-753b-47d6-a7c9-7490cec38297 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:00.300436 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b999dd17-753b-47d6-a7c9-7490cec38297 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:00.341195 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-b999dd17-753b-47d6-a7c9-7490cec38297 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:00:00.343331 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-b999dd17-753b-47d6-a7c9-7490cec38297 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:00:00.399906 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-9165334f-9f8e-4b8c-9021-30eb3c9503b9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c1ba5c61edf9465580593a84cab97e9a', 'request_id': 'req-9165334f-9f8e-4b8c-9021-30eb3c9503b9'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:00.414941 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-9165334f-9f8e-4b8c-9021-30eb3c9503b9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c1ba5c61edf9465580593a84cab97e9a', 'request_id': 'req-9165334f-9f8e-4b8c-9021-30eb3c9503b9'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:00.450313 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-9165334f-9f8e-4b8c-9021-30eb3c9503b9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c1ba5c61edf9465580593a84cab97e9a', 'request_id': 'req-9165334f-9f8e-4b8c-9021-30eb3c9503b9'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:00.457313 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 960/1911] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 16:00:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:00:00.461708 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b999dd17-753b-47d6-a7c9-7490cec38297 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:00.471510 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b999dd17-753b-47d6-a7c9-7490cec38297 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:00.472386 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-9165334f-9f8e-4b8c-9021-30eb3c9503b9 admin admin] The token cache is being invalidate because project c1ba5c61edf9465580593a84cab97e9a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81512) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 16:00:00.472684 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-9165334f-9f8e-4b8c-9021-30eb3c9503b9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:00.476422 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7507a54b-5837-4fa0-9939-0c2666c039b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:00.477442 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7507a54b-5837-4fa0-9939-0c2666c039b6 admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:00.477567 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7507a54b-5837-4fa0-9939-0c2666c039b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:00.477818 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7507a54b-5837-4fa0-9939-0c2666c039b6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:00.478551 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7507a54b-5837-4fa0-9939-0c2666c039b6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:00:00.478911 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7507a54b-5837-4fa0-9939-0c2666c039b6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:00.480848 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7507a54b-5837-4fa0-9939-0c2666c039b6 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:00.481789 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-9165334f-9f8e-4b8c-9021-30eb3c9503b9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:00.485935 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 952/1912] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Aug 30 16:00:00 2022] DELETE /identity/v3/projects/c1ba5c61edf9465580593a84cab97e9a => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 16:00:00.505682 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 961/1913] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Aug 30 16:00:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:00:00.509618 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7507a54b-5837-4fa0-9939-0c2666c039b6 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:00.514436 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bec6fcd-76b2-4217-94aa-fd9e8db45a2f None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:00.515037 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bec6fcd-76b2-4217-94aa-fd9e8db45a2f None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:00.516133 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bec6fcd-76b2-4217-94aa-fd9e8db45a2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:00.516601 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-7507a54b-5837-4fa0-9939-0c2666c039b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:00:00.546323 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7507a54b-5837-4fa0-9939-0c2666c039b6 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:00.551674 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c020aca-26e6-4bc3-a36a-14c76c6f6dc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:00.552367 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c020aca-26e6-4bc3-a36a-14c76c6f6dc5 admin admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:00.552888 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c020aca-26e6-4bc3-a36a-14c76c6f6dc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:00.553311 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c020aca-26e6-4bc3-a36a-14c76c6f6dc5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:00.554045 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c020aca-26e6-4bc3-a36a-14c76c6f6dc5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:00.556524 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c020aca-26e6-4bc3-a36a-14c76c6f6dc5 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:00.561124 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-6bec6fcd-76b2-4217-94aa-fd9e8db45a2f None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:00:00.563001 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-6bec6fcd-76b2-4217-94aa-fd9e8db45a2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:00:00.575556 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 953/1914] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 16:00:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:00:00.585114 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f263c815-2b67-44e6-abf1-c2f4cd7ae7d0 None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:00.585723 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f263c815-2b67-44e6-abf1-c2f4cd7ae7d0 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:00.586129 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f263c815-2b67-44e6-abf1-c2f4cd7ae7d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:00.589367 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 962/1915] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Aug 30 16:00:00 2022] POST /identity/v3/projects => generated 377 bytes in 80 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:00:00.593268 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c020aca-26e6-4bc3-a36a-14c76c6f6dc5 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:00.611223 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-f263c815-2b67-44e6-abf1-c2f4cd7ae7d0 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:00:00.612936 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-f263c815-2b67-44e6-abf1-c2f4cd7ae7d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:00:00.675705 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:00:00.684596 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c020aca-26e6-4bc3-a36a-14c76c6f6dc5 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:00.695721 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bda15f-5051-490e-9a51-ffe1091d9d5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:00.696897 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bda15f-5051-490e-9a51-ffe1091d9d5e admin admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:00.697455 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bda15f-5051-490e-9a51-ffe1091d9d5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:00.697939 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bda15f-5051-490e-9a51-ffe1091d9d5e admin admin] PATH_INFO: `/v3/users` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:00.698787 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6bda15f-5051-490e-9a51-ffe1091d9d5e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:00.700971 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6bda15f-5051-490e-9a51-ffe1091d9d5e admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:00.704941 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 954/1916] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 16:00:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 16:00:00.709378 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f263c815-2b67-44e6-abf1-c2f4cd7ae7d0 None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:00.722374 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f263c815-2b67-44e6-abf1-c2f4cd7ae7d0 None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:00.728765 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 963/1917] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Aug 30 16:00:00 2022] POST /identity/v3/users => generated 289 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:00:00.729817 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5851a8-7517-466a-95d7-6effd78c440f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:00.730656 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5851a8-7517-466a-95d7-6effd78c440f admin admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:00.732295 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5851a8-7517-466a-95d7-6effd78c440f admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:00.732733 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5851a8-7517-466a-95d7-6effd78c440f admin admin] PATH_INFO: `/v3/domains` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:00.733732 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5851a8-7517-466a-95d7-6effd78c440f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81512) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:00:00.733867 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bda15f-5051-490e-9a51-ffe1091d9d5e admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:00.734754 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5851a8-7517-466a-95d7-6effd78c440f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:00.736686 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5851a8-7517-466a-95d7-6effd78c440f admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:00.748538 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bda15f-5051-490e-9a51-ffe1091d9d5e admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:00.753687 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 955/1918] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Aug 30 16:00:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:00:00.754650 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9ff162-b369-42a0-a788-6ee11197af36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:00.755322 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9ff162-b369-42a0-a788-6ee11197af36 admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:00.755656 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9ff162-b369-42a0-a788-6ee11197af36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:00.756095 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9ff162-b369-42a0-a788-6ee11197af36 admin admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:00.757010 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9ff162-b369-42a0-a788-6ee11197af36 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:00.757912 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5851a8-7517-466a-95d7-6effd78c440f admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:00.759165 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9ff162-b369-42a0-a788-6ee11197af36 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:00.769299 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5851a8-7517-466a-95d7-6effd78c440f admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:00.773401 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 964/1919] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 30 16:00:00 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 16:00:00.776364 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9ff162-b369-42a0-a788-6ee11197af36 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:00.776903 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5051eacd-fef2-4f34-a9b6-d86e75393dc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:00.777693 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5051eacd-fef2-4f34-a9b6-d86e75393dc5 admin admin] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:00.778070 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5051eacd-fef2-4f34-a9b6-d86e75393dc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:00.778434 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5051eacd-fef2-4f34-a9b6-d86e75393dc5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:00.779179 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5051eacd-fef2-4f34-a9b6-d86e75393dc5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:00.781308 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5051eacd-fef2-4f34-a9b6-d86e75393dc5 admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:00.787355 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9ff162-b369-42a0-a788-6ee11197af36 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:00.792150 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085da8c3-accf-4117-aa61-22d7da22799d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:00.792913 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085da8c3-accf-4117-aa61-22d7da22799d admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:00.793190 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085da8c3-accf-4117-aa61-22d7da22799d admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:00.794003 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085da8c3-accf-4117-aa61-22d7da22799d admin admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:00.794146 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085da8c3-accf-4117-aa61-22d7da22799d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:00.796173 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085da8c3-accf-4117-aa61-22d7da22799d admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:00.809692 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 965/1920] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 30 16:00:00 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:00:00.814114 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085da8c3-accf-4117-aa61-22d7da22799d admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:00.818158 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 956/1921] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Aug 30 16:00:00 2022] POST /identity/v3/projects => generated 387 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:00:00.822475 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5051eacd-fef2-4f34-a9b6-d86e75393dc5 admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:00.825606 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085da8c3-accf-4117-aa61-22d7da22799d admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:00.844032 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:DELETE=1,keystone:SELECT=95,keystone:INSERT=4 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:00:00.908663 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8682abce-f0c8-4d96-adb3-85e76c61419f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:00.909551 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8682abce-f0c8-4d96-adb3-85e76c61419f admin admin] REQUEST_METHOD: `PUT` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:00.909823 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8682abce-f0c8-4d96-adb3-85e76c61419f admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:00.910164 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8682abce-f0c8-4d96-adb3-85e76c61419f admin admin] PATH_INFO: `/v3/projects/581d1f1799394d33a5dc3948aafe092d/users/390ddf75e1e8489dbfa77e8bf93e0e3c/roles/df8f9f844b1b439181d75a07c1a93418` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:00.935180 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8682abce-f0c8-4d96-adb3-85e76c61419f admin admin] RBAC: Authorizing `identity:create_grant(project_id=581d1f1799394d33a5dc3948aafe092d, user_id=390ddf75e1e8489dbfa77e8bf93e0e3c, role_id=df8f9f844b1b439181d75a07c1a93418)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:00.939810 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8682abce-f0c8-4d96-adb3-85e76c61419f admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:00.950103 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5051eacd-fef2-4f34-a9b6-d86e75393dc5 admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:00.970222 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770e1934-3120-4643-8228-dab6c0faee77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:00.972480 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 966/1922] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 30 16:00:00 2022] PUT /identity/v3/projects/581d1f1799394d33a5dc3948aafe092d/users/390ddf75e1e8489dbfa77e8bf93e0e3c/roles/df8f9f844b1b439181d75a07c1a93418 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 16:00:00.972835 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770e1934-3120-4643-8228-dab6c0faee77 admin admin] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:00.973125 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770e1934-3120-4643-8228-dab6c0faee77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:00.973995 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770e1934-3120-4643-8228-dab6c0faee77 admin admin] PATH_INFO: `/v3/users` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:00.974116 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770e1934-3120-4643-8228-dab6c0faee77 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:00.976188 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770e1934-3120-4643-8228-dab6c0faee77 admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:00.983724 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a306b81-afec-4e20-aeb2-127b8cd0237d None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:00.984006 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a306b81-afec-4e20-aeb2-127b8cd0237d None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:00.984274 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a306b81-afec-4e20-aeb2-127b8cd0237d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:01.015497 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 957/1923] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Aug 30 16:00:00 2022] POST /identity/v3/users => generated 294 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:00:01.020477 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770e1934-3120-4643-8228-dab6c0faee77 admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:01.035207 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770e1934-3120-4643-8228-dab6c0faee77 admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:01.042661 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-5a306b81-afec-4e20-aeb2-127b8cd0237d None None] MFA Rules not processed for user `390ddf75e1e8489dbfa77e8bf93e0e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:00:01.045329 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-5a306b81-afec-4e20-aeb2-127b8cd0237d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:00:01.137652 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3add2fe8-32be-4d01-9842-7795069cf1c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:01.139829 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3add2fe8-32be-4d01-9842-7795069cf1c5 admin admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:01.139970 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3add2fe8-32be-4d01-9842-7795069cf1c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:01.140225 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3add2fe8-32be-4d01-9842-7795069cf1c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:01.140936 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3add2fe8-32be-4d01-9842-7795069cf1c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:01.149467 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3add2fe8-32be-4d01-9842-7795069cf1c5 admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:01.161662 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 967/1924] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 16:00:00 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:00:01.167915 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 958/1925] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 30 16:00:01 2022] GET /identity/v3/roles => generated 1420 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:00:01.171305 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a306b81-afec-4e20-aeb2-127b8cd0237d None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:01.174108 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3add2fe8-32be-4d01-9842-7795069cf1c5 admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:01.176116 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-3add2fe8-32be-4d01-9842-7795069cf1c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:00:01.195023 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a306b81-afec-4e20-aeb2-127b8cd0237d None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:01.200750 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6517b3db-3865-4969-bbd3-442eecc6e19d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:01.201558 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6517b3db-3865-4969-bbd3-442eecc6e19d admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:01.201833 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6517b3db-3865-4969-bbd3-442eecc6e19d admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:01.202103 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6517b3db-3865-4969-bbd3-442eecc6e19d admin admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:01.203015 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6517b3db-3865-4969-bbd3-442eecc6e19d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:01.204738 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6517b3db-3865-4969-bbd3-442eecc6e19d admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:01.227009 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 968/1926] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 30 16:00:01 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:00:01.227971 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6517b3db-3865-4969-bbd3-442eecc6e19d admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:01.238358 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6517b3db-3865-4969-bbd3-442eecc6e19d admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:01.243733 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59df04d-04b6-4b23-942d-171541fab84d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:01.244557 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59df04d-04b6-4b23-942d-171541fab84d admin admin] REQUEST_METHOD: `PUT` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:01.244788 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59df04d-04b6-4b23-942d-171541fab84d admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:01.245053 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59df04d-04b6-4b23-942d-171541fab84d admin admin] PATH_INFO: `/v3/projects/a1f2ee7526c64195827c9862346e9ef3/users/d6a06cdc5f7f47e785cfa29fe5d1ac22/roles/df8f9f844b1b439181d75a07c1a93418` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:01.250351 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:00:01.256656 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3add2fe8-32be-4d01-9842-7795069cf1c5 admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:01.269496 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59df04d-04b6-4b23-942d-171541fab84d admin admin] RBAC: Authorizing `identity:create_grant(project_id=a1f2ee7526c64195827c9862346e9ef3, user_id=d6a06cdc5f7f47e785cfa29fe5d1ac22, role_id=df8f9f844b1b439181d75a07c1a93418)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:01.272389 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59df04d-04b6-4b23-942d-171541fab84d admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:01.272643 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c789afd7-295f-4043-9ba9-6aa0edf2271c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0100a04821804993866ea20f159663f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:01.273740 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c789afd7-295f-4043-9ba9-6aa0edf2271c service cinder] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:01.274205 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c789afd7-295f-4043-9ba9-6aa0edf2271c service cinder] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:01.274923 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c789afd7-295f-4043-9ba9-6aa0edf2271c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:01.288667 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c789afd7-295f-4043-9ba9-6aa0edf2271c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:01.290618 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c789afd7-295f-4043-9ba9-6aa0edf2271c service cinder] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:01.297324 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 969/1927] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 30 16:00:01 2022] PUT /identity/v3/projects/a1f2ee7526c64195827c9862346e9ef3/users/d6a06cdc5f7f47e785cfa29fe5d1ac22/roles/df8f9f844b1b439181d75a07c1a93418 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:00:01.300315 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4784ad9-99e2-4f0a-b150-e83f05f8e8e5 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:01.300601 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4784ad9-99e2-4f0a-b150-e83f05f8e8e5 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:01.300871 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4784ad9-99e2-4f0a-b150-e83f05f8e8e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:01.301432 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 959/1928] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 16:00:01 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 16:00:01.334239 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-c4784ad9-99e2-4f0a-b150-e83f05f8e8e5 None None] MFA Rules not processed for user `d6a06cdc5f7f47e785cfa29fe5d1ac22`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:00:01.335749 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-c4784ad9-99e2-4f0a-b150-e83f05f8e8e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:00:01.422427 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:00:01.454027 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 970/1929] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 16:00:01 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 16:00:01.455400 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c789afd7-295f-4043-9ba9-6aa0edf2271c service cinder] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:01.467446 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c789afd7-295f-4043-9ba9-6aa0edf2271c service cinder] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:01.530505 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5530e6a-d014-41ca-8043-e192b6ba862d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:01.531132 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5530e6a-d014-41ca-8043-e192b6ba862d admin admin] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:01.531401 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5530e6a-d014-41ca-8043-e192b6ba862d admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:01.531674 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5530e6a-d014-41ca-8043-e192b6ba862d admin admin] PATH_INFO: `/v3/projects` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:01.532314 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5530e6a-d014-41ca-8043-e192b6ba862d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:01.534419 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5530e6a-d014-41ca-8043-e192b6ba862d admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:01.584137 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 960/1930] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Aug 30 16:00:01 2022] POST /identity/v3/projects => generated 387 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:00:01.585535 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4784ad9-99e2-4f0a-b150-e83f05f8e8e5 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:01.600146 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4784ad9-99e2-4f0a-b150-e83f05f8e8e5 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:01.677202 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34705e0b-c085-4f7c-8a7c-fc7997f63de8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:01.677915 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34705e0b-c085-4f7c-8a7c-fc7997f63de8 admin admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:01.678332 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34705e0b-c085-4f7c-8a7c-fc7997f63de8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:01.678700 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34705e0b-c085-4f7c-8a7c-fc7997f63de8 admin admin] PATH_INFO: `/v3/users` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:01.679599 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34705e0b-c085-4f7c-8a7c-fc7997f63de8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:01.681988 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34705e0b-c085-4f7c-8a7c-fc7997f63de8 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:01.716199 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 971/1931] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Aug 30 16:00:01 2022] POST /identity/v3/users => generated 294 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:00:01.719953 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5530e6a-d014-41ca-8043-e192b6ba862d admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:01.731248 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5530e6a-d014-41ca-8043-e192b6ba862d admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:01.736832 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e54f49b-e95e-4bd5-a674-7b3d9922a5f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:01.737651 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e54f49b-e95e-4bd5-a674-7b3d9922a5f7 admin admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:01.738171 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e54f49b-e95e-4bd5-a674-7b3d9922a5f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:01.738898 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e54f49b-e95e-4bd5-a674-7b3d9922a5f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:01.739854 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e54f49b-e95e-4bd5-a674-7b3d9922a5f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:01.741680 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e54f49b-e95e-4bd5-a674-7b3d9922a5f7 admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:01.762904 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 961/1932] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 30 16:00:01 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:00:01.767686 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34705e0b-c085-4f7c-8a7c-fc7997f63de8 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:01.789937 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34705e0b-c085-4f7c-8a7c-fc7997f63de8 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:01.795639 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e5be83-83ff-450e-9b0d-7550d637112c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:01.796461 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e5be83-83ff-450e-9b0d-7550d637112c admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:01.796913 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e5be83-83ff-450e-9b0d-7550d637112c admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:01.797371 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e5be83-83ff-450e-9b0d-7550d637112c admin admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:01.798173 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e5be83-83ff-450e-9b0d-7550d637112c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:01.799939 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e5be83-83ff-450e-9b0d-7550d637112c admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:01.812172 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 972/1933] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 30 16:00:01 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:00:01.816944 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e54f49b-e95e-4bd5-a674-7b3d9922a5f7 admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:01.851328 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e54f49b-e95e-4bd5-a674-7b3d9922a5f7 admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:01.867188 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef97be08-9a6e-4728-837a-e548e4ee4644 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:01.867997 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef97be08-9a6e-4728-837a-e548e4ee4644 admin admin] REQUEST_METHOD: `PUT` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:01.868432 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef97be08-9a6e-4728-837a-e548e4ee4644 admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:01.868786 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef97be08-9a6e-4728-837a-e548e4ee4644 admin admin] PATH_INFO: `/v3/projects/81926b88fbd843c99fbeab73330e5d22/users/8e545661a40340569b9e8c491c32a171/roles/df8f9f844b1b439181d75a07c1a93418` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:01.909733 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef97be08-9a6e-4728-837a-e548e4ee4644 admin admin] RBAC: Authorizing `identity:create_grant(project_id=81926b88fbd843c99fbeab73330e5d22, user_id=8e545661a40340569b9e8c491c32a171, role_id=df8f9f844b1b439181d75a07c1a93418)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:01.913862 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef97be08-9a6e-4728-837a-e548e4ee4644 admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:01.931143 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 962/1934] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 30 16:00:01 2022] PUT /identity/v3/projects/81926b88fbd843c99fbeab73330e5d22/users/8e545661a40340569b9e8c491c32a171/roles/df8f9f844b1b439181d75a07c1a93418 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 16:00:01.941944 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3287b3f-a241-4731-aa83-122fc2d6470a None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:01.942540 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3287b3f-a241-4731-aa83-122fc2d6470a None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:01.942540 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3287b3f-a241-4731-aa83-122fc2d6470a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:02.001787 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-e3287b3f-a241-4731-aa83-122fc2d6470a None None] MFA Rules not processed for user `8e545661a40340569b9e8c491c32a171`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:00:02.003247 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-e3287b3f-a241-4731-aa83-122fc2d6470a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:00:02.082981 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 973/1935] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 16:00:01 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 16:00:02.087576 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef97be08-9a6e-4728-837a-e548e4ee4644 admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:02.152261 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef97be08-9a6e-4728-837a-e548e4ee4644 admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:02.157699 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abacb14d-aad2-46fb-a14b-850bf0835638 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:02.158301 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abacb14d-aad2-46fb-a14b-850bf0835638 admin admin] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:02.158578 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abacb14d-aad2-46fb-a14b-850bf0835638 admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:02.158847 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abacb14d-aad2-46fb-a14b-850bf0835638 admin admin] PATH_INFO: `/v3/projects` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:02.159956 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abacb14d-aad2-46fb-a14b-850bf0835638 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:02.161931 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abacb14d-aad2-46fb-a14b-850bf0835638 admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:02.194323 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 963/1936] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Aug 30 16:00:02 2022] POST /identity/v3/projects => generated 387 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:00:02.196485 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3287b3f-a241-4731-aa83-122fc2d6470a None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:02.209765 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3287b3f-a241-4731-aa83-122fc2d6470a None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:02.277091 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:00:02.284931 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae3befd-b29a-4145-a503-5660abb28427 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:02.285978 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae3befd-b29a-4145-a503-5660abb28427 admin admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:02.286485 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae3befd-b29a-4145-a503-5660abb28427 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:02.287004 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae3befd-b29a-4145-a503-5660abb28427 admin admin] PATH_INFO: `/v3/users` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:02.287679 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae3befd-b29a-4145-a503-5660abb28427 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:02.290290 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae3befd-b29a-4145-a503-5660abb28427 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:02.318124 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 974/1937] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Aug 30 16:00:02 2022] POST /identity/v3/users => generated 300 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:00:02.319469 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abacb14d-aad2-46fb-a14b-850bf0835638 admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:02.332717 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abacb14d-aad2-46fb-a14b-850bf0835638 admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:02.337576 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9cb81a-3831-4bb1-8b54-68119e7c1f54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:02.338183 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9cb81a-3831-4bb1-8b54-68119e7c1f54 admin admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:02.338466 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9cb81a-3831-4bb1-8b54-68119e7c1f54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:02.338739 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9cb81a-3831-4bb1-8b54-68119e7c1f54 admin admin] PATH_INFO: `/v3/roles` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:02.339453 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9cb81a-3831-4bb1-8b54-68119e7c1f54 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:02.341050 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9cb81a-3831-4bb1-8b54-68119e7c1f54 admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:02.353248 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 964/1938] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 30 16:00:02 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:00:02.361884 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae3befd-b29a-4145-a503-5660abb28427 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:02.372556 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae3befd-b29a-4145-a503-5660abb28427 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:02.377765 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e58b1b-47f9-441f-a882-27950b300301 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:02.378517 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e58b1b-47f9-441f-a882-27950b300301 admin admin] REQUEST_METHOD: `PUT` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:02.378797 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e58b1b-47f9-441f-a882-27950b300301 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:02.379143 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e58b1b-47f9-441f-a882-27950b300301 admin admin] PATH_INFO: `/v3/projects/4bef8b7d26094d059f7bfb995b28c1c7/users/133ec84b1a13419db01dc44545a8b1bb/roles/f265386950a641ac8413b53eda4ae34d` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:02.395467 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e58b1b-47f9-441f-a882-27950b300301 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4bef8b7d26094d059f7bfb995b28c1c7, user_id=133ec84b1a13419db01dc44545a8b1bb, role_id=f265386950a641ac8413b53eda4ae34d)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:02.398235 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e58b1b-47f9-441f-a882-27950b300301 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:02.412512 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 975/1939] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 30 16:00:02 2022] PUT /identity/v3/projects/4bef8b7d26094d059f7bfb995b28c1c7/users/133ec84b1a13419db01dc44545a8b1bb/roles/f265386950a641ac8413b53eda4ae34d => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:00:02.416652 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9cb81a-3831-4bb1-8b54-68119e7c1f54 admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:02.430456 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9cb81a-3831-4bb1-8b54-68119e7c1f54 admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:02.439870 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:00:02.492394 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec20b26-0912-4d0b-b3cb-dec54256722a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:02.493008 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec20b26-0912-4d0b-b3cb-dec54256722a admin admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:02.493320 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec20b26-0912-4d0b-b3cb-dec54256722a admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:02.493595 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec20b26-0912-4d0b-b3cb-dec54256722a admin admin] PATH_INFO: `/v3/roles` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:02.494241 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec20b26-0912-4d0b-b3cb-dec54256722a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:02.495907 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec20b26-0912-4d0b-b3cb-dec54256722a admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:02.506353 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 965/1940] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 30 16:00:02 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:00:02.510774 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e58b1b-47f9-441f-a882-27950b300301 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:02.521690 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e58b1b-47f9-441f-a882-27950b300301 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:02.527477 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d739d7-1489-447d-af1c-f3beae527e25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:02.528381 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d739d7-1489-447d-af1c-f3beae527e25 admin admin] REQUEST_METHOD: `PUT` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:02.528646 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d739d7-1489-447d-af1c-f3beae527e25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:02.529593 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d739d7-1489-447d-af1c-f3beae527e25 admin admin] PATH_INFO: `/v3/projects/4bef8b7d26094d059f7bfb995b28c1c7/users/133ec84b1a13419db01dc44545a8b1bb/roles/f265386950a641ac8413b53eda4ae34d` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:02.530951 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d739d7-1489-447d-af1c-f3beae527e25 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4bef8b7d26094d059f7bfb995b28c1c7, user_id=133ec84b1a13419db01dc44545a8b1bb, role_id=f265386950a641ac8413b53eda4ae34d)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:02.533625 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d739d7-1489-447d-af1c-f3beae527e25 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:02.546517 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 976/1941] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 30 16:00:02 2022] PUT /identity/v3/projects/4bef8b7d26094d059f7bfb995b28c1c7/users/133ec84b1a13419db01dc44545a8b1bb/roles/f265386950a641ac8413b53eda4ae34d => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:00:02.557745 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776e5575-5d6c-48af-b299-f9513ee5b0d9 None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:02.558068 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776e5575-5d6c-48af-b299-f9513ee5b0d9 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:02.558335 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776e5575-5d6c-48af-b299-f9513ee5b0d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:02.593317 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-776e5575-5d6c-48af-b299-f9513ee5b0d9 None None] MFA Rules not processed for user `133ec84b1a13419db01dc44545a8b1bb`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:00:02.595433 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-776e5575-5d6c-48af-b299-f9513ee5b0d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:00:02.684577 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 966/1942] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 16:00:02 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:00:02.696093 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d739d7-1489-447d-af1c-f3beae527e25 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:02.698124 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-30d739d7-1489-447d-af1c-f3beae527e25 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:00:02.757703 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d739d7-1489-447d-af1c-f3beae527e25 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:02.761999 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4c8792-ea58-4183-8f81-481b934982ad service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0100a04821804993866ea20f159663f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:02.762297 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4c8792-ea58-4183-8f81-481b934982ad service cinder] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:02.762566 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4c8792-ea58-4183-8f81-481b934982ad service cinder] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:02.762829 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4c8792-ea58-4183-8f81-481b934982ad service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:02.773383 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4c8792-ea58-4183-8f81-481b934982ad service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:02.774836 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4c8792-ea58-4183-8f81-481b934982ad service cinder] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:02.785433 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 977/1943] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 16:00:02 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:00:03.558906 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776e5575-5d6c-48af-b299-f9513ee5b0d9 None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:03.568375 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776e5575-5d6c-48af-b299-f9513ee5b0d9 None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:03.571968 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81785554-d3d7-43c0-9e19-5d56c53f7d6b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0100a04821804993866ea20f159663f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:03.572806 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81785554-d3d7-43c0-9e19-5d56c53f7d6b service cinder] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:03.572954 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81785554-d3d7-43c0-9e19-5d56c53f7d6b service cinder] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:03.573395 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81785554-d3d7-43c0-9e19-5d56c53f7d6b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:03.633096 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81785554-d3d7-43c0-9e19-5d56c53f7d6b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:03.634868 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81785554-d3d7-43c0-9e19-5d56c53f7d6b service cinder] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:03.645063 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 967/1944] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 16:00:03 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:00:12.782086 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=45,keystone:INSERT=5 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:00:13.641386 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=85 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:00:19.541989 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4c8792-ea58-4183-8f81-481b934982ad service cinder] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:19.550729 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4c8792-ea58-4183-8f81-481b934982ad service cinder] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:19.554595 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8cf502-84eb-4948-b9bf-df140a1571bd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0100a04821804993866ea20f159663f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:19.555203 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8cf502-84eb-4948-b9bf-df140a1571bd service cinder] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:19.555474 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8cf502-84eb-4948-b9bf-df140a1571bd service cinder] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:19.555739 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8cf502-84eb-4948-b9bf-df140a1571bd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:19.622645 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8cf502-84eb-4948-b9bf-df140a1571bd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:19.624220 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8cf502-84eb-4948-b9bf-df140a1571bd service cinder] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:19.633811 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 978/1945] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 16:00:19 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:00:25.137432 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81785554-d3d7-43c0-9e19-5d56c53f7d6b service cinder] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:25.222155 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81785554-d3d7-43c0-9e19-5d56c53f7d6b service cinder] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:25.227133 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83b11f8-7007-4550-b215-462e639beddc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:25.227764 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83b11f8-7007-4550-b215-462e639beddc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:25.228048 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83b11f8-7007-4550-b215-462e639beddc admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:25.228307 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83b11f8-7007-4550-b215-462e639beddc admin admin] PATH_INFO: `/v3/users/d6a06cdc5f7f47e785cfa29fe5d1ac22` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:25.229614 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83b11f8-7007-4550-b215-462e639beddc admin admin] RBAC: Authorizing `identity:delete_user(user_id=d6a06cdc5f7f47e785cfa29fe5d1ac22)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:25.231633 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83b11f8-7007-4550-b215-462e639beddc admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:25.283224 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-b83b11f8-7007-4550-b215-462e639beddc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd6a06cdc5f7f47e785cfa29fe5d1ac22'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:25.292336 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-b83b11f8-7007-4550-b215-462e639beddc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd6a06cdc5f7f47e785cfa29fe5d1ac22'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:25.315488 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-b83b11f8-7007-4550-b215-462e639beddc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd6a06cdc5f7f47e785cfa29fe5d1ac22'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:25.324188 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-b83b11f8-7007-4550-b215-462e639beddc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd6a06cdc5f7f47e785cfa29fe5d1ac22'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:25.325317 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-b83b11f8-7007-4550-b215-462e639beddc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd6a06cdc5f7f47e785cfa29fe5d1ac22'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:25.333054 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 968/1946] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Aug 30 16:00:25 2022] DELETE /identity/v3/users/d6a06cdc5f7f47e785cfa29fe5d1ac22 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:00:25.339195 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264dd6bd-d475-4754-817a-c4338cbafd33 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:25.339765 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264dd6bd-d475-4754-817a-c4338cbafd33 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:25.340618 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264dd6bd-d475-4754-817a-c4338cbafd33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:25.366257 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-264dd6bd-d475-4754-817a-c4338cbafd33 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:00:25.368536 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-264dd6bd-d475-4754-817a-c4338cbafd33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:00:25.505988 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 979/1947] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 16:00:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:00:25.518376 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83b11f8-7007-4550-b215-462e639beddc admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:25.520227 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-b83b11f8-7007-4550-b215-462e639beddc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:00:25.607132 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83b11f8-7007-4550-b215-462e639beddc admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:25.612172 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14b00dd-e811-42d0-8e2a-fe89b65d771f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f4157105f3488f807b7475008ab722', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:25.613014 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14b00dd-e811-42d0-8e2a-fe89b65d771f service neutron] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:25.613773 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14b00dd-e811-42d0-8e2a-fe89b65d771f service neutron] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:25.614063 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14b00dd-e811-42d0-8e2a-fe89b65d771f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:25.624980 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14b00dd-e811-42d0-8e2a-fe89b65d771f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:25.626820 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14b00dd-e811-42d0-8e2a-fe89b65d771f service neutron] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:25.639114 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 969/1948] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 16:00:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:00:26.309254 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264dd6bd-d475-4754-817a-c4338cbafd33 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:26.310828 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-264dd6bd-d475-4754-817a-c4338cbafd33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:00:26.370899 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264dd6bd-d475-4754-817a-c4338cbafd33 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:26.380080 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00fb1e6-9454-48b3-96c9-5233e7ada862 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:26.380826 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00fb1e6-9454-48b3-96c9-5233e7ada862 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:26.381167 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00fb1e6-9454-48b3-96c9-5233e7ada862 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:26.381535 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00fb1e6-9454-48b3-96c9-5233e7ada862 admin admin] PATH_INFO: `/v3/projects/a1f2ee7526c64195827c9862346e9ef3` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:26.382745 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00fb1e6-9454-48b3-96c9-5233e7ada862 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a1f2ee7526c64195827c9862346e9ef3)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:26.384786 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00fb1e6-9454-48b3-96c9-5233e7ada862 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:26.483362 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-c00fb1e6-9454-48b3-96c9-5233e7ada862 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a1f2ee7526c64195827c9862346e9ef3', 'request_id': 'req-c00fb1e6-9454-48b3-96c9-5233e7ada862'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:26.517730 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-c00fb1e6-9454-48b3-96c9-5233e7ada862 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a1f2ee7526c64195827c9862346e9ef3', 'request_id': 'req-c00fb1e6-9454-48b3-96c9-5233e7ada862'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:26.525853 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-c00fb1e6-9454-48b3-96c9-5233e7ada862 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a1f2ee7526c64195827c9862346e9ef3', 'request_id': 'req-c00fb1e6-9454-48b3-96c9-5233e7ada862'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:26.540643 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-c00fb1e6-9454-48b3-96c9-5233e7ada862 admin admin] The token cache is being invalidate because project a1f2ee7526c64195827c9862346e9ef3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81511) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 16:00:26.540998 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-c00fb1e6-9454-48b3-96c9-5233e7ada862 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:26.541641 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-c00fb1e6-9454-48b3-96c9-5233e7ada862 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:26.543253 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 980/1949] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Aug 30 16:00:26 2022] DELETE /identity/v3/projects/a1f2ee7526c64195827c9862346e9ef3 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:00:26.547960 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14b00dd-e811-42d0-8e2a-fe89b65d771f service neutron] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:26.559645 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14b00dd-e811-42d0-8e2a-fe89b65d771f service neutron] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:26.580231 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:00:26.628377 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ccc093-92bb-4ced-ba32-6a287abd1d6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:26.629057 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ccc093-92bb-4ced-ba32-6a287abd1d6a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:26.629416 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ccc093-92bb-4ced-ba32-6a287abd1d6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:26.629686 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ccc093-92bb-4ced-ba32-6a287abd1d6a admin admin] PATH_INFO: `/v3/users/8e545661a40340569b9e8c491c32a171` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:26.631067 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ccc093-92bb-4ced-ba32-6a287abd1d6a admin admin] RBAC: Authorizing `identity:delete_user(user_id=8e545661a40340569b9e8c491c32a171)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:26.633008 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ccc093-92bb-4ced-ba32-6a287abd1d6a admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:26.698332 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-23ccc093-92bb-4ced-ba32-6a287abd1d6a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8e545661a40340569b9e8c491c32a171'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:26.707252 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-23ccc093-92bb-4ced-ba32-6a287abd1d6a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8e545661a40340569b9e8c491c32a171'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:26.721932 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-23ccc093-92bb-4ced-ba32-6a287abd1d6a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8e545661a40340569b9e8c491c32a171'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:26.733891 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-23ccc093-92bb-4ced-ba32-6a287abd1d6a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8e545661a40340569b9e8c491c32a171'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:26.734613 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-23ccc093-92bb-4ced-ba32-6a287abd1d6a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8e545661a40340569b9e8c491c32a171'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:26.742933 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 970/1950] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Aug 30 16:00:26 2022] DELETE /identity/v3/users/8e545661a40340569b9e8c491c32a171 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:00:27.103124 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00fb1e6-9454-48b3-96c9-5233e7ada862 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:27.105759 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-c00fb1e6-9454-48b3-96c9-5233e7ada862 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:00:27.122985 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00fb1e6-9454-48b3-96c9-5233e7ada862 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:27.158132 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:00:27.204282 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b8ea61-49ff-47b6-9a21-35d29381b9cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:27.204966 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b8ea61-49ff-47b6-9a21-35d29381b9cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:27.205266 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b8ea61-49ff-47b6-9a21-35d29381b9cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:27.205542 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b8ea61-49ff-47b6-9a21-35d29381b9cf admin admin] PATH_INFO: `/v3/projects/81926b88fbd843c99fbeab73330e5d22` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:27.206666 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b8ea61-49ff-47b6-9a21-35d29381b9cf admin admin] RBAC: Authorizing `identity:delete_project(project_id=81926b88fbd843c99fbeab73330e5d22)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:27.208584 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b8ea61-49ff-47b6-9a21-35d29381b9cf admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:27.258528 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-64b8ea61-49ff-47b6-9a21-35d29381b9cf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '81926b88fbd843c99fbeab73330e5d22', 'request_id': 'req-64b8ea61-49ff-47b6-9a21-35d29381b9cf'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:27.267801 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-64b8ea61-49ff-47b6-9a21-35d29381b9cf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '81926b88fbd843c99fbeab73330e5d22', 'request_id': 'req-64b8ea61-49ff-47b6-9a21-35d29381b9cf'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:27.273887 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-64b8ea61-49ff-47b6-9a21-35d29381b9cf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '81926b88fbd843c99fbeab73330e5d22', 'request_id': 'req-64b8ea61-49ff-47b6-9a21-35d29381b9cf'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:27.283634 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-64b8ea61-49ff-47b6-9a21-35d29381b9cf admin admin] The token cache is being invalidate because project 81926b88fbd843c99fbeab73330e5d22 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81511) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 16:00:27.283936 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-64b8ea61-49ff-47b6-9a21-35d29381b9cf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:27.284539 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-64b8ea61-49ff-47b6-9a21-35d29381b9cf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:27.286049 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 981/1951] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Aug 30 16:00:27 2022] DELETE /identity/v3/projects/81926b88fbd843c99fbeab73330e5d22 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:00:27.290470 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ccc093-92bb-4ced-ba32-6a287abd1d6a admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:27.292376 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-23ccc093-92bb-4ced-ba32-6a287abd1d6a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:00:27.306795 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ccc093-92bb-4ced-ba32-6a287abd1d6a admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:27.368564 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1713dcc3-e45d-4c0f-bc9c-3d9f05d8416b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:27.369555 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1713dcc3-e45d-4c0f-bc9c-3d9f05d8416b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:27.369860 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1713dcc3-e45d-4c0f-bc9c-3d9f05d8416b admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:27.370131 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1713dcc3-e45d-4c0f-bc9c-3d9f05d8416b admin admin] PATH_INFO: `/v3/users/133ec84b1a13419db01dc44545a8b1bb` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:27.371469 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1713dcc3-e45d-4c0f-bc9c-3d9f05d8416b admin admin] RBAC: Authorizing `identity:delete_user(user_id=133ec84b1a13419db01dc44545a8b1bb)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:27.373435 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1713dcc3-e45d-4c0f-bc9c-3d9f05d8416b admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:27.422543 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:00:27.425182 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-1713dcc3-e45d-4c0f-bc9c-3d9f05d8416b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '133ec84b1a13419db01dc44545a8b1bb'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:27.434819 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-1713dcc3-e45d-4c0f-bc9c-3d9f05d8416b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '133ec84b1a13419db01dc44545a8b1bb'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:27.446756 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-1713dcc3-e45d-4c0f-bc9c-3d9f05d8416b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '133ec84b1a13419db01dc44545a8b1bb'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:27.453577 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-1713dcc3-e45d-4c0f-bc9c-3d9f05d8416b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '133ec84b1a13419db01dc44545a8b1bb'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:27.454269 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-1713dcc3-e45d-4c0f-bc9c-3d9f05d8416b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '133ec84b1a13419db01dc44545a8b1bb'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:27.460029 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 971/1952] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Aug 30 16:00:27 2022] DELETE /identity/v3/users/133ec84b1a13419db01dc44545a8b1bb => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:00:27.778491 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b8ea61-49ff-47b6-9a21-35d29381b9cf admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:27.780866 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-64b8ea61-49ff-47b6-9a21-35d29381b9cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:00:27.798624 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b8ea61-49ff-47b6-9a21-35d29381b9cf admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:27.860615 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683ed08d-75b1-4ce7-995c-70865fee2f79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:27.861277 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683ed08d-75b1-4ce7-995c-70865fee2f79 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:27.861565 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683ed08d-75b1-4ce7-995c-70865fee2f79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:27.861838 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683ed08d-75b1-4ce7-995c-70865fee2f79 admin admin] PATH_INFO: `/v3/projects/4bef8b7d26094d059f7bfb995b28c1c7` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:27.862964 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683ed08d-75b1-4ce7-995c-70865fee2f79 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4bef8b7d26094d059f7bfb995b28c1c7)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:27.865290 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683ed08d-75b1-4ce7-995c-70865fee2f79 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:27.915151 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-683ed08d-75b1-4ce7-995c-70865fee2f79 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4bef8b7d26094d059f7bfb995b28c1c7', 'request_id': 'req-683ed08d-75b1-4ce7-995c-70865fee2f79'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:27.919192 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:00:27.924852 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-683ed08d-75b1-4ce7-995c-70865fee2f79 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4bef8b7d26094d059f7bfb995b28c1c7', 'request_id': 'req-683ed08d-75b1-4ce7-995c-70865fee2f79'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:27.934825 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-683ed08d-75b1-4ce7-995c-70865fee2f79 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4bef8b7d26094d059f7bfb995b28c1c7', 'request_id': 'req-683ed08d-75b1-4ce7-995c-70865fee2f79'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:27.946221 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-683ed08d-75b1-4ce7-995c-70865fee2f79 admin admin] The token cache is being invalidate because project 4bef8b7d26094d059f7bfb995b28c1c7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81511) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 16:00:27.946622 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-683ed08d-75b1-4ce7-995c-70865fee2f79 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:27.947280 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-683ed08d-75b1-4ce7-995c-70865fee2f79 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:27.949135 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 982/1953] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Aug 30 16:00:27 2022] DELETE /identity/v3/projects/4bef8b7d26094d059f7bfb995b28c1c7 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:00:27.955770 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce120c7-be22-44f1-a67d-2dcc8dfddc30 None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:27.956278 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce120c7-be22-44f1-a67d-2dcc8dfddc30 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:27.956371 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce120c7-be22-44f1-a67d-2dcc8dfddc30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:27.976994 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-dce120c7-be22-44f1-a67d-2dcc8dfddc30 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:00:27.978310 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-dce120c7-be22-44f1-a67d-2dcc8dfddc30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:00:28.053369 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 972/1954] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 16:00:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:00:28.062194 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f4d581-ce9c-4a63-a4bd-ed6573316d07 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:28.062686 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f4d581-ce9c-4a63-a4bd-ed6573316d07 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:28.063140 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f4d581-ce9c-4a63-a4bd-ed6573316d07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:28.091506 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-f3f4d581-ce9c-4a63-a4bd-ed6573316d07 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:00:28.092871 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-f3f4d581-ce9c-4a63-a4bd-ed6573316d07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:00:28.102417 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 983/1955] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 16:00:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 16:00:28.107024 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce120c7-be22-44f1-a67d-2dcc8dfddc30 None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:28.117256 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce120c7-be22-44f1-a67d-2dcc8dfddc30 None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:28.122162 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213ab391-25b0-4fe8-84e6-df17e7e2998e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:28.122766 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213ab391-25b0-4fe8-84e6-df17e7e2998e admin admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:28.123041 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213ab391-25b0-4fe8-84e6-df17e7e2998e admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:28.123308 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213ab391-25b0-4fe8-84e6-df17e7e2998e admin admin] PATH_INFO: `/v3/domains` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:28.123951 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213ab391-25b0-4fe8-84e6-df17e7e2998e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81512) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:00:28.124271 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213ab391-25b0-4fe8-84e6-df17e7e2998e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:28.125915 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213ab391-25b0-4fe8-84e6-df17e7e2998e admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:28.141253 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 973/1956] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Aug 30 16:00:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:00:28.142489 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f4d581-ce9c-4a63-a4bd-ed6573316d07 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:28.152271 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f4d581-ce9c-4a63-a4bd-ed6573316d07 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:28.157655 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9a83d9-2c07-4c9b-8c36-b0cd0df99b6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:28.158628 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9a83d9-2c07-4c9b-8c36-b0cd0df99b6f admin admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:28.159032 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9a83d9-2c07-4c9b-8c36-b0cd0df99b6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:28.159305 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9a83d9-2c07-4c9b-8c36-b0cd0df99b6f admin admin] PATH_INFO: `/v3/projects` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:28.159944 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9a83d9-2c07-4c9b-8c36-b0cd0df99b6f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:28.162628 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9a83d9-2c07-4c9b-8c36-b0cd0df99b6f admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:28.181366 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213ab391-25b0-4fe8-84e6-df17e7e2998e admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:28.183403 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-213ab391-25b0-4fe8-84e6-df17e7e2998e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:00:28.201376 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 984/1957] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Aug 30 16:00:28 2022] POST /identity/v3/projects => generated 401 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:00:28.201532 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213ab391-25b0-4fe8-84e6-df17e7e2998e admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:28.202786 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9a83d9-2c07-4c9b-8c36-b0cd0df99b6f admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:28.263420 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e444de8-6b86-49a9-bce5-e28930005be1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:28.264170 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e444de8-6b86-49a9-bce5-e28930005be1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:28.264553 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e444de8-6b86-49a9-bce5-e28930005be1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:28.265139 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e444de8-6b86-49a9-bce5-e28930005be1 admin admin] PATH_INFO: `/v3/users/390ddf75e1e8489dbfa77e8bf93e0e3c` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:28.266505 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e444de8-6b86-49a9-bce5-e28930005be1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=390ddf75e1e8489dbfa77e8bf93e0e3c)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:28.268552 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e444de8-6b86-49a9-bce5-e28930005be1 admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:28.269408 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9a83d9-2c07-4c9b-8c36-b0cd0df99b6f admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:28.274449 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1d87e5-f2c1-4b86-99a1-b0520ff148e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:28.275403 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1d87e5-f2c1-4b86-99a1-b0520ff148e8 admin admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:28.275403 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1d87e5-f2c1-4b86-99a1-b0520ff148e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:28.275618 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1d87e5-f2c1-4b86-99a1-b0520ff148e8 admin admin] PATH_INFO: `/v3/users` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:28.276285 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1d87e5-f2c1-4b86-99a1-b0520ff148e8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:28.278341 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1d87e5-f2c1-4b86-99a1-b0520ff148e8 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:28.309010 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 985/1958] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Aug 30 16:00:28 2022] POST /identity/v3/users => generated 301 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 16:00:28.325405 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1d87e5-f2c1-4b86-99a1-b0520ff148e8 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:28.334308 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-4e444de8-6b86-49a9-bce5-e28930005be1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '390ddf75e1e8489dbfa77e8bf93e0e3c'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:28.337431 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:DELETE=11,keystone:SELECT=87,keystone:INSERT=2 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:00:28.339461 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1d87e5-f2c1-4b86-99a1-b0520ff148e8 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:28.345083 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-4e444de8-6b86-49a9-bce5-e28930005be1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '390ddf75e1e8489dbfa77e8bf93e0e3c'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:28.346830 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e8ea8e-71b5-4246-b19b-a61c9f75a5e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:28.348426 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e8ea8e-71b5-4246-b19b-a61c9f75a5e8 admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:28.348426 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e8ea8e-71b5-4246-b19b-a61c9f75a5e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:28.348426 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e8ea8e-71b5-4246-b19b-a61c9f75a5e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:28.348822 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e8ea8e-71b5-4246-b19b-a61c9f75a5e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:28.350792 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e8ea8e-71b5-4246-b19b-a61c9f75a5e8 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:28.360829 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-4e444de8-6b86-49a9-bce5-e28930005be1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '390ddf75e1e8489dbfa77e8bf93e0e3c'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:28.363663 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 986/1959] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 30 16:00:28 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:00:28.367673 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e8ea8e-71b5-4246-b19b-a61c9f75a5e8 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:28.369348 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-4e444de8-6b86-49a9-bce5-e28930005be1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '390ddf75e1e8489dbfa77e8bf93e0e3c'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:28.369956 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-4e444de8-6b86-49a9-bce5-e28930005be1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '390ddf75e1e8489dbfa77e8bf93e0e3c'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:28.376239 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 974/1960] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Aug 30 16:00:28 2022] DELETE /identity/v3/users/390ddf75e1e8489dbfa77e8bf93e0e3c => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 16:00:28.379146 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e8ea8e-71b5-4246-b19b-a61c9f75a5e8 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:28.380528 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a71a92-4a87-44e4-bf49-48765c315554 None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:28.380821 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a71a92-4a87-44e4-bf49-48765c315554 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:28.381106 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a71a92-4a87-44e4-bf49-48765c315554 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:28.401636 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-69a71a92-4a87-44e4-bf49-48765c315554 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:00:28.402863 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-69a71a92-4a87-44e4-bf49-48765c315554 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:00:28.446491 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c152f7f-a59f-4155-a552-77dfb4f797c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:28.447351 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c152f7f-a59f-4155-a552-77dfb4f797c3 admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:28.447628 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c152f7f-a59f-4155-a552-77dfb4f797c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:28.447893 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c152f7f-a59f-4155-a552-77dfb4f797c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:28.448534 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c152f7f-a59f-4155-a552-77dfb4f797c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:28.450164 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c152f7f-a59f-4155-a552-77dfb4f797c3 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:28.460017 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 987/1961] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 30 16:00:28 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 16:00:28.464138 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c152f7f-a59f-4155-a552-77dfb4f797c3 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:28.465815 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-0c152f7f-a59f-4155-a552-77dfb4f797c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:00:28.468294 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 975/1962] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 16:00:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:00:28.477844 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a71a92-4a87-44e4-bf49-48765c315554 None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:28.478214 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c152f7f-a59f-4155-a552-77dfb4f797c3 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:28.479190 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-69a71a92-4a87-44e4-bf49-48765c315554 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:00:28.482834 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fa2360-51f7-49f2-ae7f-ad011c7a1901 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:28.483629 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fa2360-51f7-49f2-ae7f-ad011c7a1901 admin admin] REQUEST_METHOD: `PUT` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:28.483915 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fa2360-51f7-49f2-ae7f-ad011c7a1901 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:28.484313 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fa2360-51f7-49f2-ae7f-ad011c7a1901 admin admin] PATH_INFO: `/v3/projects/7d207512d3604da9b045ac5b5d52158f/users/54969ba34af24f79bebaf13e8ece491d/roles/df8f9f844b1b439181d75a07c1a93418` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:28.499296 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fa2360-51f7-49f2-ae7f-ad011c7a1901 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7d207512d3604da9b045ac5b5d52158f, user_id=54969ba34af24f79bebaf13e8ece491d, role_id=df8f9f844b1b439181d75a07c1a93418)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:28.502024 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fa2360-51f7-49f2-ae7f-ad011c7a1901 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:28.517274 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 988/1963] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 30 16:00:28 2022] PUT /identity/v3/projects/7d207512d3604da9b045ac5b5d52158f/users/54969ba34af24f79bebaf13e8ece491d/roles/df8f9f844b1b439181d75a07c1a93418 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 16:00:28.529528 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f175252a-fb3e-402c-9b60-f785384dba89 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:28.529735 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f175252a-fb3e-402c-9b60-f785384dba89 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:28.530003 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f175252a-fb3e-402c-9b60-f785384dba89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:28.538005 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=93,keystone:DELETE=2,keystone:INSERT=5 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:00:28.538377 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a71a92-4a87-44e4-bf49-48765c315554 None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:28.560803 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-f175252a-fb3e-402c-9b60-f785384dba89 None None] MFA Rules not processed for user `54969ba34af24f79bebaf13e8ece491d`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:00:28.562093 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-f175252a-fb3e-402c-9b60-f785384dba89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:00:28.588241 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b62b384-d1ce-40e4-ac5c-195385101263 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f4157105f3488f807b7475008ab722', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:28.588851 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b62b384-d1ce-40e4-ac5c-195385101263 service neutron] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:28.589124 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b62b384-d1ce-40e4-ac5c-195385101263 service neutron] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:28.589422 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b62b384-d1ce-40e4-ac5c-195385101263 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:28.631626 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 989/1964] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 16:00:28 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 16:00:28.636169 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f175252a-fb3e-402c-9b60-f785384dba89 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:28.640374 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:DELETE=3,keystone:SELECT=97 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:00:28.666133 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b62b384-d1ce-40e4-ac5c-195385101263 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:28.667591 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b62b384-d1ce-40e4-ac5c-195385101263 service neutron] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:28.677057 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 976/1965] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 16:00:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 16:00:28.698675 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f175252a-fb3e-402c-9b60-f785384dba89 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:28.703628 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d188b03-86fb-4980-9b22-24da4576694c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:28.704258 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d188b03-86fb-4980-9b22-24da4576694c admin admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:28.704531 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d188b03-86fb-4980-9b22-24da4576694c admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:28.704790 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d188b03-86fb-4980-9b22-24da4576694c admin admin] PATH_INFO: `/v3/projects` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:28.705466 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d188b03-86fb-4980-9b22-24da4576694c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:28.707370 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d188b03-86fb-4980-9b22-24da4576694c admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:28.739849 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 990/1966] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Aug 30 16:00:28 2022] POST /identity/v3/projects => generated 399 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:00:28.743850 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b62b384-d1ce-40e4-ac5c-195385101263 service neutron] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:28.755536 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b62b384-d1ce-40e4-ac5c-195385101263 service neutron] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:28.828703 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aeb6752-e18c-4722-b8e7-2398a2b799ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:28.829458 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aeb6752-e18c-4722-b8e7-2398a2b799ef admin admin] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:28.829821 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aeb6752-e18c-4722-b8e7-2398a2b799ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:28.830151 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aeb6752-e18c-4722-b8e7-2398a2b799ef admin admin] PATH_INFO: `/v3/users` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:28.830852 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aeb6752-e18c-4722-b8e7-2398a2b799ef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:28.833068 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aeb6752-e18c-4722-b8e7-2398a2b799ef admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:28.862627 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 977/1967] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Aug 30 16:00:28 2022] POST /identity/v3/users => generated 306 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:00:28.864527 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d188b03-86fb-4980-9b22-24da4576694c admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:28.875625 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d188b03-86fb-4980-9b22-24da4576694c admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:28.880263 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f64145-1f93-47d7-a6b2-a22467bae40b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:28.880879 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f64145-1f93-47d7-a6b2-a22467bae40b admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:28.881159 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f64145-1f93-47d7-a6b2-a22467bae40b admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:28.881467 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f64145-1f93-47d7-a6b2-a22467bae40b admin admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:28.882147 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f64145-1f93-47d7-a6b2-a22467bae40b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:28.883782 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f64145-1f93-47d7-a6b2-a22467bae40b admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:28.894357 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 991/1968] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 30 16:00:28 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:00:28.898529 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aeb6752-e18c-4722-b8e7-2398a2b799ef admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:28.908949 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aeb6752-e18c-4722-b8e7-2398a2b799ef admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:28.913978 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2083e866-56a8-49a7-948e-dabada876722 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:28.915177 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2083e866-56a8-49a7-948e-dabada876722 admin admin] REQUEST_METHOD: `PUT` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:28.915461 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2083e866-56a8-49a7-948e-dabada876722 admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:28.915734 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2083e866-56a8-49a7-948e-dabada876722 admin admin] PATH_INFO: `/v3/projects/dce940000998408da011b8830bcf1f93/users/80ccdd176c01419fa8469e39f0399519/roles/f265386950a641ac8413b53eda4ae34d` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:28.949092 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2083e866-56a8-49a7-948e-dabada876722 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dce940000998408da011b8830bcf1f93, user_id=80ccdd176c01419fa8469e39f0399519, role_id=f265386950a641ac8413b53eda4ae34d)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:28.951854 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2083e866-56a8-49a7-948e-dabada876722 admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:28.958076 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f64145-1f93-47d7-a6b2-a22467bae40b admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:28.960039 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-12f64145-1f93-47d7-a6b2-a22467bae40b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:00:28.969285 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 978/1969] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 30 16:00:28 2022] PUT /identity/v3/projects/dce940000998408da011b8830bcf1f93/users/80ccdd176c01419fa8469e39f0399519/roles/f265386950a641ac8413b53eda4ae34d => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:00:28.973774 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f64145-1f93-47d7-a6b2-a22467bae40b admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:28.974072 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2083e866-56a8-49a7-948e-dabada876722 admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:29.031840 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9775b13f-8298-4cd0-8851-900653ae597e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:29.032488 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9775b13f-8298-4cd0-8851-900653ae597e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:29.032768 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9775b13f-8298-4cd0-8851-900653ae597e admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:29.033041 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9775b13f-8298-4cd0-8851-900653ae597e admin admin] PATH_INFO: `/v3/projects/581d1f1799394d33a5dc3948aafe092d` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:29.034285 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9775b13f-8298-4cd0-8851-900653ae597e admin admin] RBAC: Authorizing `identity:delete_project(project_id=581d1f1799394d33a5dc3948aafe092d)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:29.036147 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9775b13f-8298-4cd0-8851-900653ae597e admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:29.039833 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2083e866-56a8-49a7-948e-dabada876722 admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:29.044879 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433b60a7-b20f-4657-94ab-f2195d1f08ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:29.045523 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433b60a7-b20f-4657-94ab-f2195d1f08ce admin admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:29.046022 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433b60a7-b20f-4657-94ab-f2195d1f08ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:29.046307 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433b60a7-b20f-4657-94ab-f2195d1f08ce admin admin] PATH_INFO: `/v3/roles` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:29.046992 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433b60a7-b20f-4657-94ab-f2195d1f08ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:29.047462 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:00:29.048747 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433b60a7-b20f-4657-94ab-f2195d1f08ce admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:29.059112 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 979/1970] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 30 16:00:28 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 16:00:29.063217 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433b60a7-b20f-4657-94ab-f2195d1f08ce admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:29.074987 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433b60a7-b20f-4657-94ab-f2195d1f08ce admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:29.079947 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f909a5-9f75-46d4-ba2a-d1b86b0bde14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:29.080592 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f909a5-9f75-46d4-ba2a-d1b86b0bde14 admin admin] REQUEST_METHOD: `PUT` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:29.080865 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f909a5-9f75-46d4-ba2a-d1b86b0bde14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:29.081130 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f909a5-9f75-46d4-ba2a-d1b86b0bde14 admin admin] PATH_INFO: `/v3/projects/dce940000998408da011b8830bcf1f93/users/80ccdd176c01419fa8469e39f0399519/roles/f265386950a641ac8413b53eda4ae34d` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:29.083027 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f909a5-9f75-46d4-ba2a-d1b86b0bde14 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dce940000998408da011b8830bcf1f93, user_id=80ccdd176c01419fa8469e39f0399519, role_id=f265386950a641ac8413b53eda4ae34d)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:29.085728 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f909a5-9f75-46d4-ba2a-d1b86b0bde14 admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:29.090430 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-9775b13f-8298-4cd0-8851-900653ae597e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '581d1f1799394d33a5dc3948aafe092d', 'request_id': 'req-9775b13f-8298-4cd0-8851-900653ae597e'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:29.096237 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 980/1971] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 30 16:00:29 2022] PUT /identity/v3/projects/dce940000998408da011b8830bcf1f93/users/80ccdd176c01419fa8469e39f0399519/roles/f265386950a641ac8413b53eda4ae34d => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 16:00:29.098969 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-9775b13f-8298-4cd0-8851-900653ae597e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '581d1f1799394d33a5dc3948aafe092d', 'request_id': 'req-9775b13f-8298-4cd0-8851-900653ae597e'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:29.106307 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-9775b13f-8298-4cd0-8851-900653ae597e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '581d1f1799394d33a5dc3948aafe092d', 'request_id': 'req-9775b13f-8298-4cd0-8851-900653ae597e'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:29.107142 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9486b216-e8bb-48b9-9c9e-a92275164cd2 None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:29.107416 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9486b216-e8bb-48b9-9c9e-a92275164cd2 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:29.107699 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9486b216-e8bb-48b9-9c9e-a92275164cd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:29.120145 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-9775b13f-8298-4cd0-8851-900653ae597e admin admin] The token cache is being invalidate because project 581d1f1799394d33a5dc3948aafe092d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81511) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 16:00:29.120434 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-9775b13f-8298-4cd0-8851-900653ae597e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:29.121097 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-9775b13f-8298-4cd0-8851-900653ae597e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:29.126465 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 992/1972] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Aug 30 16:00:28 2022] DELETE /identity/v3/projects/581d1f1799394d33a5dc3948aafe092d => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:00:29.127462 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0635a2c0-049f-4c11-a90b-d89fb6577760 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:29.127684 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0635a2c0-049f-4c11-a90b-d89fb6577760 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:29.127955 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0635a2c0-049f-4c11-a90b-d89fb6577760 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:29.146176 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-9486b216-e8bb-48b9-9c9e-a92275164cd2 None None] MFA Rules not processed for user `80ccdd176c01419fa8469e39f0399519`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:00:29.147415 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-9486b216-e8bb-48b9-9c9e-a92275164cd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:00:29.149552 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-0635a2c0-049f-4c11-a90b-d89fb6577760 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:00:29.150727 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-0635a2c0-049f-4c11-a90b-d89fb6577760 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:00:29.157939 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:00:29.218824 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 993/1973] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 16:00:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 16:00:29.220523 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf0c839-5e35-4652-ba92-8990c2321161 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:29.220873 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf0c839-5e35-4652-ba92-8990c2321161 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:29.221211 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf0c839-5e35-4652-ba92-8990c2321161 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:29.248022 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-5bf0c839-5e35-4652-ba92-8990c2321161 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:00:29.249800 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-5bf0c839-5e35-4652-ba92-8990c2321161 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:00:29.252595 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 981/1974] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 16:00:29 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 16:00:29.264218 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9486b216-e8bb-48b9-9c9e-a92275164cd2 None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:29.266152 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-9486b216-e8bb-48b9-9c9e-a92275164cd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:00:29.267580 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 994/1975] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 16:00:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 16:00:29.268458 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf0c839-5e35-4652-ba92-8990c2321161 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:29.279614 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf0c839-5e35-4652-ba92-8990c2321161 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:29.285303 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9889c0fd-6c64-4bb8-b8a7-1ddd75fa54ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:29.285869 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9889c0fd-6c64-4bb8-b8a7-1ddd75fa54ec admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:29.286150 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9889c0fd-6c64-4bb8-b8a7-1ddd75fa54ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:29.286602 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9889c0fd-6c64-4bb8-b8a7-1ddd75fa54ec admin admin] PATH_INFO: `/v3/domains` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:29.287100 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9889c0fd-6c64-4bb8-b8a7-1ddd75fa54ec admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:00:29.287359 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9889c0fd-6c64-4bb8-b8a7-1ddd75fa54ec admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:29.288965 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9889c0fd-6c64-4bb8-b8a7-1ddd75fa54ec admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:29.308689 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 995/1976] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Aug 30 16:00:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:00:29.309785 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9889c0fd-6c64-4bb8-b8a7-1ddd75fa54ec admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:29.319683 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9889c0fd-6c64-4bb8-b8a7-1ddd75fa54ec admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:29.324374 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42738100-32aa-4f68-b600-ba68cf480910 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:29.324964 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42738100-32aa-4f68-b600-ba68cf480910 admin admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:29.325265 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42738100-32aa-4f68-b600-ba68cf480910 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:29.325541 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42738100-32aa-4f68-b600-ba68cf480910 admin admin] PATH_INFO: `/v3/projects` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:29.325786 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9486b216-e8bb-48b9-9c9e-a92275164cd2 None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:29.326161 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42738100-32aa-4f68-b600-ba68cf480910 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:29.328103 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42738100-32aa-4f68-b600-ba68cf480910 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:29.329373 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fd6431-1e65-4571-8f57-73c797bf90c9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0100a04821804993866ea20f159663f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:29.329988 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fd6431-1e65-4571-8f57-73c797bf90c9 service cinder] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:29.330264 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fd6431-1e65-4571-8f57-73c797bf90c9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:29.330534 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fd6431-1e65-4571-8f57-73c797bf90c9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:29.339357 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17fd6431-1e65-4571-8f57-73c797bf90c9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:29.340793 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17fd6431-1e65-4571-8f57-73c797bf90c9 service cinder] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:29.353147 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 982/1977] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 16:00:29 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 16:00:29.358742 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 996/1978] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Aug 30 16:00:29 2022] POST /identity/v3/projects => generated 375 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 16:00:29.363398 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fd6431-1e65-4571-8f57-73c797bf90c9 service cinder] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:29.373826 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fd6431-1e65-4571-8f57-73c797bf90c9 service cinder] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:29.431762 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde019b8-c94c-449e-b0aa-22c05e853c56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:29.432384 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde019b8-c94c-449e-b0aa-22c05e853c56 admin admin] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:29.432660 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde019b8-c94c-449e-b0aa-22c05e853c56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:29.432930 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde019b8-c94c-449e-b0aa-22c05e853c56 admin admin] PATH_INFO: `/v3/users` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:29.433630 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde019b8-c94c-449e-b0aa-22c05e853c56 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:29.435552 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde019b8-c94c-449e-b0aa-22c05e853c56 admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:29.435752 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42738100-32aa-4f68-b600-ba68cf480910 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:29.462802 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 983/1979] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Aug 30 16:00:29 2022] POST /identity/v3/users => generated 288 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:00:29.464288 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde019b8-c94c-449e-b0aa-22c05e853c56 admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:29.475205 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde019b8-c94c-449e-b0aa-22c05e853c56 admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:29.484748 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a57f94-9922-460e-9398-00d3ea876f66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:29.485461 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a57f94-9922-460e-9398-00d3ea876f66 admin admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:29.485923 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a57f94-9922-460e-9398-00d3ea876f66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:29.486445 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a57f94-9922-460e-9398-00d3ea876f66 admin admin] PATH_INFO: `/v3/roles` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:29.487630 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a57f94-9922-460e-9398-00d3ea876f66 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:29.490942 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a57f94-9922-460e-9398-00d3ea876f66 admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:29.505363 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 984/1980] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 30 16:00:29 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 16:00:29.516093 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a57f94-9922-460e-9398-00d3ea876f66 admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:29.520750 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42738100-32aa-4f68-b600-ba68cf480910 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:29.524816 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b999ace7-da8e-49c2-945a-057491482aea service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0100a04821804993866ea20f159663f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:29.525468 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b999ace7-da8e-49c2-945a-057491482aea service cinder] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:29.525746 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b999ace7-da8e-49c2-945a-057491482aea service cinder] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:29.526013 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b999ace7-da8e-49c2-945a-057491482aea service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:29.527764 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-b999ace7-da8e-49c2-945a-057491482aea service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:00:29.529289 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a57f94-9922-460e-9398-00d3ea876f66 admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:29.533658 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7dd90f-93dd-4ad5-a37a-340fc5c2dc2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:29.534273 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7dd90f-93dd-4ad5-a37a-340fc5c2dc2d admin admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:29.534552 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7dd90f-93dd-4ad5-a37a-340fc5c2dc2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:29.534826 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7dd90f-93dd-4ad5-a37a-340fc5c2dc2d admin admin] PATH_INFO: `/v3/roles` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:29.535575 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7dd90f-93dd-4ad5-a37a-340fc5c2dc2d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:29.537087 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7dd90f-93dd-4ad5-a37a-340fc5c2dc2d admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:29.541572 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:00:29.549360 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 985/1981] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 30 16:00:29 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:00:29.553948 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7dd90f-93dd-4ad5-a37a-340fc5c2dc2d admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:29.564675 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7dd90f-93dd-4ad5-a37a-340fc5c2dc2d admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:29.570209 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6216ba9-4bb3-4243-b2df-763866a49e85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:29.570940 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6216ba9-4bb3-4243-b2df-763866a49e85 admin admin] REQUEST_METHOD: `PUT` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:29.571305 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6216ba9-4bb3-4243-b2df-763866a49e85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:29.571662 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6216ba9-4bb3-4243-b2df-763866a49e85 admin admin] PATH_INFO: `/v3/projects/058ab65d5dcb4bea82fe83721c58a1c6/users/af9f01e25ee947db9c11cc4577e980ea/roles/df8f9f844b1b439181d75a07c1a93418` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:29.589655 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6216ba9-4bb3-4243-b2df-763866a49e85 admin admin] RBAC: Authorizing `identity:create_grant(project_id=058ab65d5dcb4bea82fe83721c58a1c6, user_id=af9f01e25ee947db9c11cc4577e980ea, role_id=df8f9f844b1b439181d75a07c1a93418)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:29.592433 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6216ba9-4bb3-4243-b2df-763866a49e85 admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:29.594831 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:00:29.603393 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b999ace7-da8e-49c2-945a-057491482aea service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:29.604953 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b999ace7-da8e-49c2-945a-057491482aea service cinder] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:29.610609 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 986/1982] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 30 16:00:29 2022] PUT /identity/v3/projects/058ab65d5dcb4bea82fe83721c58a1c6/users/af9f01e25ee947db9c11cc4577e980ea/roles/df8f9f844b1b439181d75a07c1a93418 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 16:00:29.617832 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 997/1983] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 16:00:29 2022] GET /identity/v3/auth/tokens => generated 3278 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:00:29.621770 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a66f840-5218-440e-b97a-f7b10e5e6b41 None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:29.622065 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a66f840-5218-440e-b97a-f7b10e5e6b41 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:29.622340 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a66f840-5218-440e-b97a-f7b10e5e6b41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:29.656431 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-2a66f840-5218-440e-b97a-f7b10e5e6b41 None None] MFA Rules not processed for user `af9f01e25ee947db9c11cc4577e980ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:00:29.657797 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-2a66f840-5218-440e-b97a-f7b10e5e6b41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:00:29.722474 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 987/1984] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 16:00:29 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:00:29.733747 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b999ace7-da8e-49c2-945a-057491482aea service cinder] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:29.735166 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-b999ace7-da8e-49c2-945a-057491482aea service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:00:29.746940 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b999ace7-da8e-49c2-945a-057491482aea service cinder] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:29.804565 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51170551-2f7d-47c2-8ebc-a3fc1f1ae24d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0100a04821804993866ea20f159663f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:29.805237 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51170551-2f7d-47c2-8ebc-a3fc1f1ae24d service cinder] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:29.805524 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51170551-2f7d-47c2-8ebc-a3fc1f1ae24d service cinder] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:29.805794 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51170551-2f7d-47c2-8ebc-a3fc1f1ae24d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:29.820319 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51170551-2f7d-47c2-8ebc-a3fc1f1ae24d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:29.821854 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51170551-2f7d-47c2-8ebc-a3fc1f1ae24d service cinder] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:29.836449 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 998/1985] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 16:00:29 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:00:32.362682 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209db0b4-7f70-47f3-aa51-9eabf1e59983 None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:32.363513 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209db0b4-7f70-47f3-aa51-9eabf1e59983 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:32.363513 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209db0b4-7f70-47f3-aa51-9eabf1e59983 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:32.383958 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-209db0b4-7f70-47f3-aa51-9eabf1e59983 None None] MFA Rules not processed for user `af9f01e25ee947db9c11cc4577e980ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:00:32.385239 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-209db0b4-7f70-47f3-aa51-9eabf1e59983 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:00:32.397346 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 988/1986] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 16:00:32 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:00:35.068930 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51170551-2f7d-47c2-8ebc-a3fc1f1ae24d service cinder] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:35.071081 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-51170551-2f7d-47c2-8ebc-a3fc1f1ae24d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:00:35.166639 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51170551-2f7d-47c2-8ebc-a3fc1f1ae24d service cinder] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:35.172227 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3cbc3c-9db4-4145-a9db-154c984e52b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:35.173135 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3cbc3c-9db4-4145-a9db-154c984e52b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:35.173518 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3cbc3c-9db4-4145-a9db-154c984e52b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:35.173936 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3cbc3c-9db4-4145-a9db-154c984e52b9 admin admin] PATH_INFO: `/v3/users/54969ba34af24f79bebaf13e8ece491d` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:35.208963 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3cbc3c-9db4-4145-a9db-154c984e52b9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=54969ba34af24f79bebaf13e8ece491d)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:35.227113 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3cbc3c-9db4-4145-a9db-154c984e52b9 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:35.353084 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-fb3cbc3c-9db4-4145-a9db-154c984e52b9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '54969ba34af24f79bebaf13e8ece491d'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:35.378048 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:00:35.392155 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-fb3cbc3c-9db4-4145-a9db-154c984e52b9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '54969ba34af24f79bebaf13e8ece491d'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:35.421852 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-fb3cbc3c-9db4-4145-a9db-154c984e52b9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '54969ba34af24f79bebaf13e8ece491d'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:35.426218 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-fb3cbc3c-9db4-4145-a9db-154c984e52b9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '54969ba34af24f79bebaf13e8ece491d'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:35.449861 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-fb3cbc3c-9db4-4145-a9db-154c984e52b9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '54969ba34af24f79bebaf13e8ece491d'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:35.465678 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 999/1987] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Aug 30 16:00:35 2022] DELETE /identity/v3/users/54969ba34af24f79bebaf13e8ece491d => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:00:35.472027 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b010bb9-bbcd-4b24-a25d-a45c4c589abd None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:35.472778 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b010bb9-bbcd-4b24-a25d-a45c4c589abd None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:35.472778 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b010bb9-bbcd-4b24-a25d-a45c4c589abd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:35.495310 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-9b010bb9-bbcd-4b24-a25d-a45c4c589abd None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:00:35.496803 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-9b010bb9-bbcd-4b24-a25d-a45c4c589abd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:00:35.576800 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3cbc3c-9db4-4145-a9db-154c984e52b9 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:35.578535 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-fb3cbc3c-9db4-4145-a9db-154c984e52b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:00:35.595273 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3cbc3c-9db4-4145-a9db-154c984e52b9 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:35.599981 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 989/1988] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 16:00:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 16:00:35.607026 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34b8946-5ca7-42fd-9817-ce75113ed957 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:35.607791 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34b8946-5ca7-42fd-9817-ce75113ed957 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:35.608184 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34b8946-5ca7-42fd-9817-ce75113ed957 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:35.608528 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34b8946-5ca7-42fd-9817-ce75113ed957 admin admin] PATH_INFO: `/v3/users/d28800d13b6049f5952cf40a19f6dc22` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:35.609839 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34b8946-5ca7-42fd-9817-ce75113ed957 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d28800d13b6049f5952cf40a19f6dc22)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:35.611230 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b010bb9-bbcd-4b24-a25d-a45c4c589abd None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:35.612716 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-9b010bb9-bbcd-4b24-a25d-a45c4c589abd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:00:35.616846 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34b8946-5ca7-42fd-9817-ce75113ed957 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:35.696448 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-c34b8946-5ca7-42fd-9817-ce75113ed957 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd28800d13b6049f5952cf40a19f6dc22'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:35.697309 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b010bb9-bbcd-4b24-a25d-a45c4c589abd None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:35.701729 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b670bab-ece4-451f-9d6b-708547b47806 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f4157105f3488f807b7475008ab722', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:35.702546 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b670bab-ece4-451f-9d6b-708547b47806 service neutron] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:35.702986 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b670bab-ece4-451f-9d6b-708547b47806 service neutron] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:35.703405 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b670bab-ece4-451f-9d6b-708547b47806 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:35.710009 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-c34b8946-5ca7-42fd-9817-ce75113ed957 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd28800d13b6049f5952cf40a19f6dc22'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:35.715881 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b670bab-ece4-451f-9d6b-708547b47806 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:35.718282 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b670bab-ece4-451f-9d6b-708547b47806 service neutron] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:35.720494 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-c34b8946-5ca7-42fd-9817-ce75113ed957 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd28800d13b6049f5952cf40a19f6dc22'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:35.721786 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-c34b8946-5ca7-42fd-9817-ce75113ed957 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd28800d13b6049f5952cf40a19f6dc22'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:35.731778 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 990/1989] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 16:00:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 16:00:35.731993 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-c34b8946-5ca7-42fd-9817-ce75113ed957 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd28800d13b6049f5952cf40a19f6dc22'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:35.740699 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1000/1990] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Aug 30 16:00:35 2022] DELETE /identity/v3/users/d28800d13b6049f5952cf40a19f6dc22 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:00:35.746048 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5198404f-212b-4b5c-a933-c3c9eec5bb2e None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:35.746333 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5198404f-212b-4b5c-a933-c3c9eec5bb2e None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:35.746610 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5198404f-212b-4b5c-a933-c3c9eec5bb2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:35.798243 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-5198404f-212b-4b5c-a933-c3c9eec5bb2e None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:00:35.799505 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-5198404f-212b-4b5c-a933-c3c9eec5bb2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:00:35.821486 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:00:35.889127 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 991/1991] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 16:00:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:00:35.916302 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34b8946-5ca7-42fd-9817-ce75113ed957 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:35.918444 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-c34b8946-5ca7-42fd-9817-ce75113ed957 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:00:36.059985 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34b8946-5ca7-42fd-9817-ce75113ed957 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:36.069380 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d3904d-6283-40e4-82d4-06d3b75601a7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f4157105f3488f807b7475008ab722', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:36.071124 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d3904d-6283-40e4-82d4-06d3b75601a7 service neutron] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:36.071510 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d3904d-6283-40e4-82d4-06d3b75601a7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:36.071788 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d3904d-6283-40e4-82d4-06d3b75601a7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:36.102863 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d3904d-6283-40e4-82d4-06d3b75601a7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:36.106051 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d3904d-6283-40e4-82d4-06d3b75601a7 service neutron] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:36.120244 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1001/1992] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 16:00:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:00:36.321517 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5198404f-212b-4b5c-a933-c3c9eec5bb2e None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:36.322842 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-5198404f-212b-4b5c-a933-c3c9eec5bb2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:00:36.339336 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5198404f-212b-4b5c-a933-c3c9eec5bb2e None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:36.349868 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd249a8b-9536-435a-8966-278a2f5169f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:36.350929 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd249a8b-9536-435a-8966-278a2f5169f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:36.351358 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd249a8b-9536-435a-8966-278a2f5169f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:36.351863 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd249a8b-9536-435a-8966-278a2f5169f9 admin admin] PATH_INFO: `/v3/projects/7d207512d3604da9b045ac5b5d52158f` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:36.353269 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd249a8b-9536-435a-8966-278a2f5169f9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7d207512d3604da9b045ac5b5d52158f)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:36.355709 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd249a8b-9536-435a-8966-278a2f5169f9 admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:36.495895 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-fd249a8b-9536-435a-8966-278a2f5169f9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7d207512d3604da9b045ac5b5d52158f', 'request_id': 'req-fd249a8b-9536-435a-8966-278a2f5169f9'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:36.513422 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-fd249a8b-9536-435a-8966-278a2f5169f9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7d207512d3604da9b045ac5b5d52158f', 'request_id': 'req-fd249a8b-9536-435a-8966-278a2f5169f9'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:36.524778 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-fd249a8b-9536-435a-8966-278a2f5169f9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7d207512d3604da9b045ac5b5d52158f', 'request_id': 'req-fd249a8b-9536-435a-8966-278a2f5169f9'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:36.536824 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-fd249a8b-9536-435a-8966-278a2f5169f9 admin admin] The token cache is being invalidate because project 7d207512d3604da9b045ac5b5d52158f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81512) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 16:00:36.537587 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-fd249a8b-9536-435a-8966-278a2f5169f9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:36.538451 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-fd249a8b-9536-435a-8966-278a2f5169f9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:36.540493 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 992/1993] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Aug 30 16:00:36 2022] DELETE /identity/v3/projects/7d207512d3604da9b045ac5b5d52158f => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:00:36.544569 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d3904d-6283-40e4-82d4-06d3b75601a7 service neutron] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:36.566249 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d3904d-6283-40e4-82d4-06d3b75601a7 service neutron] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:36.647141 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:00:36.661582 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980fb0ce-dca3-46fa-a7d2-609ba60a11da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:36.662481 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980fb0ce-dca3-46fa-a7d2-609ba60a11da admin admin] REQUEST_METHOD: `DELETE` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:36.663048 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980fb0ce-dca3-46fa-a7d2-609ba60a11da admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:36.663387 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980fb0ce-dca3-46fa-a7d2-609ba60a11da admin admin] PATH_INFO: `/v3/users/80ccdd176c01419fa8469e39f0399519` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:36.664997 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980fb0ce-dca3-46fa-a7d2-609ba60a11da admin admin] RBAC: Authorizing `identity:delete_user(user_id=80ccdd176c01419fa8469e39f0399519)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:36.667909 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980fb0ce-dca3-46fa-a7d2-609ba60a11da admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:36.747728 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-980fb0ce-dca3-46fa-a7d2-609ba60a11da admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '80ccdd176c01419fa8469e39f0399519'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:36.765919 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-980fb0ce-dca3-46fa-a7d2-609ba60a11da admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '80ccdd176c01419fa8469e39f0399519'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:36.774717 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-980fb0ce-dca3-46fa-a7d2-609ba60a11da admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '80ccdd176c01419fa8469e39f0399519'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:36.778115 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-980fb0ce-dca3-46fa-a7d2-609ba60a11da admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '80ccdd176c01419fa8469e39f0399519'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:36.779319 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd249a8b-9536-435a-8966-278a2f5169f9 admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:36.780797 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-fd249a8b-9536-435a-8966-278a2f5169f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:00:36.786321 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-980fb0ce-dca3-46fa-a7d2-609ba60a11da admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '80ccdd176c01419fa8469e39f0399519'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:36.793632 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1002/1994] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Aug 30 16:00:36 2022] DELETE /identity/v3/users/80ccdd176c01419fa8469e39f0399519 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:00:36.794948 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd249a8b-9536-435a-8966-278a2f5169f9 admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:36.884100 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad57877-d0b8-4342-8ddd-c973e76e6391 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:36.884100 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad57877-d0b8-4342-8ddd-c973e76e6391 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:36.884317 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad57877-d0b8-4342-8ddd-c973e76e6391 admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:36.884494 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad57877-d0b8-4342-8ddd-c973e76e6391 admin admin] PATH_INFO: `/v3/projects/b43a2b4fe41c4d50a3c588a9c2473d63` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:36.885654 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad57877-d0b8-4342-8ddd-c973e76e6391 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b43a2b4fe41c4d50a3c588a9c2473d63)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:36.887662 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad57877-d0b8-4342-8ddd-c973e76e6391 admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:36.969530 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:00:36.973622 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-2ad57877-d0b8-4342-8ddd-c973e76e6391 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b43a2b4fe41c4d50a3c588a9c2473d63', 'request_id': 'req-2ad57877-d0b8-4342-8ddd-c973e76e6391'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:36.987854 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-2ad57877-d0b8-4342-8ddd-c973e76e6391 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b43a2b4fe41c4d50a3c588a9c2473d63', 'request_id': 'req-2ad57877-d0b8-4342-8ddd-c973e76e6391'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:36.996212 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-2ad57877-d0b8-4342-8ddd-c973e76e6391 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b43a2b4fe41c4d50a3c588a9c2473d63', 'request_id': 'req-2ad57877-d0b8-4342-8ddd-c973e76e6391'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:37.009967 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-2ad57877-d0b8-4342-8ddd-c973e76e6391 admin admin] The token cache is being invalidate because project b43a2b4fe41c4d50a3c588a9c2473d63 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81512) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 16:00:37.010306 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-2ad57877-d0b8-4342-8ddd-c973e76e6391 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:37.011790 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-2ad57877-d0b8-4342-8ddd-c973e76e6391 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:37.014598 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 993/1995] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Aug 30 16:00:36 2022] DELETE /identity/v3/projects/b43a2b4fe41c4d50a3c588a9c2473d63 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:00:37.018989 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980fb0ce-dca3-46fa-a7d2-609ba60a11da admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:37.020506 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-980fb0ce-dca3-46fa-a7d2-609ba60a11da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:00:37.034767 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980fb0ce-dca3-46fa-a7d2-609ba60a11da admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:37.097119 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad57877-d0b8-4342-8ddd-c973e76e6391 admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:37.101708 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-2ad57877-d0b8-4342-8ddd-c973e76e6391 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:00:37.129682 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da74cd79-4eb3-4496-a3f5-9c3878604dd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:37.130899 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da74cd79-4eb3-4496-a3f5-9c3878604dd0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:37.131299 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da74cd79-4eb3-4496-a3f5-9c3878604dd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:37.131577 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da74cd79-4eb3-4496-a3f5-9c3878604dd0 admin admin] PATH_INFO: `/v3/users/e6373234a7984f01aeb25925e90a0c85` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:37.132985 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da74cd79-4eb3-4496-a3f5-9c3878604dd0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e6373234a7984f01aeb25925e90a0c85)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:37.136168 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da74cd79-4eb3-4496-a3f5-9c3878604dd0 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:37.208912 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad57877-d0b8-4342-8ddd-c973e76e6391 admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:37.210423 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-da74cd79-4eb3-4496-a3f5-9c3878604dd0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e6373234a7984f01aeb25925e90a0c85'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:37.217879 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c5170a-3414-457f-b996-9fe3d868b94f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:37.232855 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c5170a-3414-457f-b996-9fe3d868b94f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:37.233084 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c5170a-3414-457f-b996-9fe3d868b94f admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:37.233389 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c5170a-3414-457f-b996-9fe3d868b94f admin admin] PATH_INFO: `/v3/projects/dce940000998408da011b8830bcf1f93` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:37.236690 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c5170a-3414-457f-b996-9fe3d868b94f admin admin] RBAC: Authorizing `identity:delete_project(project_id=dce940000998408da011b8830bcf1f93)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:37.238515 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-da74cd79-4eb3-4496-a3f5-9c3878604dd0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e6373234a7984f01aeb25925e90a0c85'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:37.239403 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c5170a-3414-457f-b996-9fe3d868b94f admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:37.243077 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:00:37.256906 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-da74cd79-4eb3-4496-a3f5-9c3878604dd0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e6373234a7984f01aeb25925e90a0c85'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:37.257902 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-da74cd79-4eb3-4496-a3f5-9c3878604dd0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e6373234a7984f01aeb25925e90a0c85'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:37.286368 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-da74cd79-4eb3-4496-a3f5-9c3878604dd0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e6373234a7984f01aeb25925e90a0c85'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:37.305490 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1003/1996] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Aug 30 16:00:37 2022] DELETE /identity/v3/users/e6373234a7984f01aeb25925e90a0c85 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:00:37.314237 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-45c5170a-3414-457f-b996-9fe3d868b94f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dce940000998408da011b8830bcf1f93', 'request_id': 'req-45c5170a-3414-457f-b996-9fe3d868b94f'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:37.330317 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-45c5170a-3414-457f-b996-9fe3d868b94f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dce940000998408da011b8830bcf1f93', 'request_id': 'req-45c5170a-3414-457f-b996-9fe3d868b94f'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:37.338959 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-45c5170a-3414-457f-b996-9fe3d868b94f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dce940000998408da011b8830bcf1f93', 'request_id': 'req-45c5170a-3414-457f-b996-9fe3d868b94f'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:37.352657 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-45c5170a-3414-457f-b996-9fe3d868b94f admin admin] The token cache is being invalidate because project dce940000998408da011b8830bcf1f93 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81512) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 16:00:37.353085 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-45c5170a-3414-457f-b996-9fe3d868b94f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:37.353886 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-45c5170a-3414-457f-b996-9fe3d868b94f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:37.355872 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 994/1997] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Aug 30 16:00:37 2022] DELETE /identity/v3/projects/dce940000998408da011b8830bcf1f93 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:00:37.360494 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699e0a2f-5365-4e63-a91f-e2084c07c850 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:37.360791 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699e0a2f-5365-4e63-a91f-e2084c07c850 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:37.361040 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699e0a2f-5365-4e63-a91f-e2084c07c850 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:37.385908 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-699e0a2f-5365-4e63-a91f-e2084c07c850 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:00:37.387339 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-699e0a2f-5365-4e63-a91f-e2084c07c850 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:00:37.457517 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1004/1998] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 16:00:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:00:37.467070 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515f246e-1c3c-45a6-96dd-c03ccc59328e None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:37.467447 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515f246e-1c3c-45a6-96dd-c03ccc59328e None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:37.467780 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515f246e-1c3c-45a6-96dd-c03ccc59328e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:37.493252 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-515f246e-1c3c-45a6-96dd-c03ccc59328e None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:00:37.494578 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-515f246e-1c3c-45a6-96dd-c03ccc59328e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:00:37.512055 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 995/1999] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 16:00:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 16:00:37.517287 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699e0a2f-5365-4e63-a91f-e2084c07c850 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:37.529340 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699e0a2f-5365-4e63-a91f-e2084c07c850 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:37.550982 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4136bf6c-3cc2-4d08-b63f-3a3d16b9913d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:37.551628 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4136bf6c-3cc2-4d08-b63f-3a3d16b9913d admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:37.551908 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4136bf6c-3cc2-4d08-b63f-3a3d16b9913d admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:37.552180 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4136bf6c-3cc2-4d08-b63f-3a3d16b9913d admin admin] PATH_INFO: `/v3/domains` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:37.552845 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4136bf6c-3cc2-4d08-b63f-3a3d16b9913d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:00:37.553144 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4136bf6c-3cc2-4d08-b63f-3a3d16b9913d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:37.554804 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4136bf6c-3cc2-4d08-b63f-3a3d16b9913d admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:37.568137 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1005/2000] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Aug 30 16:00:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 16:00:37.572607 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515f246e-1c3c-45a6-96dd-c03ccc59328e None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:37.583050 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515f246e-1c3c-45a6-96dd-c03ccc59328e None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:37.588015 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fabb9f9-1768-4a16-9116-4adcf086fb34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:37.588646 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fabb9f9-1768-4a16-9116-4adcf086fb34 admin admin] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:37.588927 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fabb9f9-1768-4a16-9116-4adcf086fb34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:37.589193 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fabb9f9-1768-4a16-9116-4adcf086fb34 admin admin] PATH_INFO: `/v3/projects` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:37.589887 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fabb9f9-1768-4a16-9116-4adcf086fb34 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:37.592433 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fabb9f9-1768-4a16-9116-4adcf086fb34 admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:37.627703 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 996/2001] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Aug 30 16:00:37 2022] POST /identity/v3/projects => generated 377 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 16:00:37.630833 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4136bf6c-3cc2-4d08-b63f-3a3d16b9913d admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:37.644941 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4136bf6c-3cc2-4d08-b63f-3a3d16b9913d admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:37.712071 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830ca0a4-32da-4fe9-8ca3-cc2d2de421c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:37.712799 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830ca0a4-32da-4fe9-8ca3-cc2d2de421c3 admin admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:37.713141 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830ca0a4-32da-4fe9-8ca3-cc2d2de421c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:37.713592 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830ca0a4-32da-4fe9-8ca3-cc2d2de421c3 admin admin] PATH_INFO: `/v3/users` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:37.714313 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830ca0a4-32da-4fe9-8ca3-cc2d2de421c3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:37.716359 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830ca0a4-32da-4fe9-8ca3-cc2d2de421c3 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:37.744358 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1006/2002] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Aug 30 16:00:37 2022] POST /identity/v3/users => generated 289 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:00:37.748516 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fabb9f9-1768-4a16-9116-4adcf086fb34 admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:37.764955 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fabb9f9-1768-4a16-9116-4adcf086fb34 admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:37.774166 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5db4f5-afaa-4da4-81a1-1a996d3dcbc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:37.774815 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5db4f5-afaa-4da4-81a1-1a996d3dcbc7 admin admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:37.775116 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5db4f5-afaa-4da4-81a1-1a996d3dcbc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:37.775372 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5db4f5-afaa-4da4-81a1-1a996d3dcbc7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:37.776696 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5db4f5-afaa-4da4-81a1-1a996d3dcbc7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:37.779219 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5db4f5-afaa-4da4-81a1-1a996d3dcbc7 admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:37.792299 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 997/2003] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 30 16:00:37 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:00:37.796698 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830ca0a4-32da-4fe9-8ca3-cc2d2de421c3 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:37.807116 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830ca0a4-32da-4fe9-8ca3-cc2d2de421c3 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:37.811880 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43afab3-5dc0-42f4-b724-b4f94ded92e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:37.812501 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43afab3-5dc0-42f4-b724-b4f94ded92e6 admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:37.812778 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43afab3-5dc0-42f4-b724-b4f94ded92e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:37.813258 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43afab3-5dc0-42f4-b724-b4f94ded92e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:37.813754 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43afab3-5dc0-42f4-b724-b4f94ded92e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:37.815714 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43afab3-5dc0-42f4-b724-b4f94ded92e6 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:37.826948 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1007/2004] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 30 16:00:37 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:00:37.831590 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5db4f5-afaa-4da4-81a1-1a996d3dcbc7 admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:37.842322 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5db4f5-afaa-4da4-81a1-1a996d3dcbc7 admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:37.847178 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afca9174-4277-4f7e-ac98-e1f3b440edde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:37.847855 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afca9174-4277-4f7e-ac98-e1f3b440edde admin admin] REQUEST_METHOD: `PUT` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:37.848134 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afca9174-4277-4f7e-ac98-e1f3b440edde admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:37.848406 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afca9174-4277-4f7e-ac98-e1f3b440edde admin admin] PATH_INFO: `/v3/projects/2374ab8ff42d4e8296fd98dce3cf46bf/users/c1243529bd0347bdb252197a225d679e/roles/df8f9f844b1b439181d75a07c1a93418` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:37.864215 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afca9174-4277-4f7e-ac98-e1f3b440edde admin admin] RBAC: Authorizing `identity:create_grant(project_id=2374ab8ff42d4e8296fd98dce3cf46bf, user_id=c1243529bd0347bdb252197a225d679e, role_id=df8f9f844b1b439181d75a07c1a93418)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:37.867092 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afca9174-4277-4f7e-ac98-e1f3b440edde admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:37.886094 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:DELETE=11,keystone:SELECT=85,keystone:INSERT=4 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:00:37.893664 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 998/2005] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 30 16:00:37 2022] PUT /identity/v3/projects/2374ab8ff42d4e8296fd98dce3cf46bf/users/c1243529bd0347bdb252197a225d679e/roles/df8f9f844b1b439181d75a07c1a93418 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 16:00:37.903774 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1ae268-9293-4c78-8d1e-55374a8e157f None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:37.904139 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1ae268-9293-4c78-8d1e-55374a8e157f None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:37.904473 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1ae268-9293-4c78-8d1e-55374a8e157f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:37.933799 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-7d1ae268-9293-4c78-8d1e-55374a8e157f None None] MFA Rules not processed for user `c1243529bd0347bdb252197a225d679e`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:00:37.935077 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-7d1ae268-9293-4c78-8d1e-55374a8e157f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:00:37.979296 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:00:38.005863 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1008/2006] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 16:00:37 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:00:38.010471 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afca9174-4277-4f7e-ac98-e1f3b440edde admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:38.080765 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afca9174-4277-4f7e-ac98-e1f3b440edde admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:38.087816 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b268a155-b398-452f-a4d9-e988425d465f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:38.088560 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b268a155-b398-452f-a4d9-e988425d465f admin admin] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:38.088838 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b268a155-b398-452f-a4d9-e988425d465f admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:38.089183 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b268a155-b398-452f-a4d9-e988425d465f admin admin] PATH_INFO: `/v3/projects` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:38.089831 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b268a155-b398-452f-a4d9-e988425d465f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:38.091943 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b268a155-b398-452f-a4d9-e988425d465f admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:38.127837 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 999/2007] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Aug 30 16:00:38 2022] POST /identity/v3/projects => generated 377 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 16:00:38.131942 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1ae268-9293-4c78-8d1e-55374a8e157f None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:38.146289 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1ae268-9293-4c78-8d1e-55374a8e157f None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:38.255723 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b268a155-b398-452f-a4d9-e988425d465f admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:38.258472 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-b268a155-b398-452f-a4d9-e988425d465f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:00:38.267594 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33114e01-12ef-4560-8b10-87083fc512a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:38.268117 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33114e01-12ef-4560-8b10-87083fc512a2 admin admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:38.268395 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33114e01-12ef-4560-8b10-87083fc512a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:38.268742 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33114e01-12ef-4560-8b10-87083fc512a2 admin admin] PATH_INFO: `/v3/users` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:38.269531 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33114e01-12ef-4560-8b10-87083fc512a2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:38.271499 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33114e01-12ef-4560-8b10-87083fc512a2 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:38.294571 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b268a155-b398-452f-a4d9-e988425d465f admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:38.299571 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1009/2008] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Aug 30 16:00:38 2022] POST /identity/v3/users => generated 289 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 16:00:38.301118 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99dc5658-3fe4-4af9-a5c5-25dade851e9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:38.301822 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dc5658-3fe4-4af9-a5c5-25dade851e9f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:38.302105 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dc5658-3fe4-4af9-a5c5-25dade851e9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:38.302586 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dc5658-3fe4-4af9-a5c5-25dade851e9f admin admin] PATH_INFO: `/v3/projects/d3a4652d80eb450a801d6aa788d94d01` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:38.303462 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33114e01-12ef-4560-8b10-87083fc512a2 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:38.303610 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99dc5658-3fe4-4af9-a5c5-25dade851e9f admin admin] RBAC: Authorizing `identity:delete_project(project_id=d3a4652d80eb450a801d6aa788d94d01)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:38.305656 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99dc5658-3fe4-4af9-a5c5-25dade851e9f admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:38.315519 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33114e01-12ef-4560-8b10-87083fc512a2 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:38.320278 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881878de-f305-4d82-80ae-5ab51712fa35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:38.320897 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881878de-f305-4d82-80ae-5ab51712fa35 admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:38.321196 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881878de-f305-4d82-80ae-5ab51712fa35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:38.321545 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881878de-f305-4d82-80ae-5ab51712fa35 admin admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:38.322223 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881878de-f305-4d82-80ae-5ab51712fa35 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:38.323825 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881878de-f305-4d82-80ae-5ab51712fa35 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:38.335525 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1010/2009] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 30 16:00:38 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 16:00:38.340027 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881878de-f305-4d82-80ae-5ab51712fa35 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:38.350889 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881878de-f305-4d82-80ae-5ab51712fa35 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:38.365817 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-99dc5658-3fe4-4af9-a5c5-25dade851e9f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd3a4652d80eb450a801d6aa788d94d01', 'request_id': 'req-99dc5658-3fe4-4af9-a5c5-25dade851e9f'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:38.378579 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-99dc5658-3fe4-4af9-a5c5-25dade851e9f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd3a4652d80eb450a801d6aa788d94d01', 'request_id': 'req-99dc5658-3fe4-4af9-a5c5-25dade851e9f'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:38.393622 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-99dc5658-3fe4-4af9-a5c5-25dade851e9f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd3a4652d80eb450a801d6aa788d94d01', 'request_id': 'req-99dc5658-3fe4-4af9-a5c5-25dade851e9f'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:38.405512 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-99dc5658-3fe4-4af9-a5c5-25dade851e9f admin admin] The token cache is being invalidate because project d3a4652d80eb450a801d6aa788d94d01 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81512) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 16:00:38.405909 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-99dc5658-3fe4-4af9-a5c5-25dade851e9f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:38.406735 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-99dc5658-3fe4-4af9-a5c5-25dade851e9f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:38.408939 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1000/2010] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Aug 30 16:00:38 2022] DELETE /identity/v3/projects/d3a4652d80eb450a801d6aa788d94d01 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:00:38.413805 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323fd98a-42c3-4e44-ab26-e1bcd5c055b2 None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:38.414163 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323fd98a-42c3-4e44-ab26-e1bcd5c055b2 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:38.414500 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323fd98a-42c3-4e44-ab26-e1bcd5c055b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:38.433181 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc05b807-bd04-45e9-a678-2bfba4788d20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:38.433929 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc05b807-bd04-45e9-a678-2bfba4788d20 admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:38.434207 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc05b807-bd04-45e9-a678-2bfba4788d20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:38.434476 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc05b807-bd04-45e9-a678-2bfba4788d20 admin admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:38.435136 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc05b807-bd04-45e9-a678-2bfba4788d20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:38.436073 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-323fd98a-42c3-4e44-ab26-e1bcd5c055b2 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:00:38.437063 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc05b807-bd04-45e9-a678-2bfba4788d20 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:38.437451 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-323fd98a-42c3-4e44-ab26-e1bcd5c055b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:00:38.446351 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1001/2011] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 16:00:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:00:38.447669 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1011/2012] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 30 16:00:38 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:00:38.451661 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323fd98a-42c3-4e44-ab26-e1bcd5c055b2 None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:38.453032 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-323fd98a-42c3-4e44-ab26-e1bcd5c055b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:00:38.454613 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4bb17c-581e-406d-a833-ddb21b861b5b None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:38.455048 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4bb17c-581e-406d-a833-ddb21b861b5b None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:38.455488 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4bb17c-581e-406d-a833-ddb21b861b5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:38.464951 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323fd98a-42c3-4e44-ab26-e1bcd5c055b2 None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:38.470137 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271e1a3a-8393-4fda-b219-b1f093b5d0c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:38.470809 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271e1a3a-8393-4fda-b219-b1f093b5d0c0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:38.471086 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271e1a3a-8393-4fda-b219-b1f093b5d0c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:38.471358 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271e1a3a-8393-4fda-b219-b1f093b5d0c0 admin admin] PATH_INFO: `/v3/projects/931345f1d3fb4d92b59750c6b979fbba/users/28b6f8d640e5403f8d50cb4bb9d50448/roles/df8f9f844b1b439181d75a07c1a93418` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:38.477100 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-df4bb17c-581e-406d-a833-ddb21b861b5b None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:00:38.478379 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-df4bb17c-581e-406d-a833-ddb21b861b5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:00:38.487610 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271e1a3a-8393-4fda-b219-b1f093b5d0c0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=931345f1d3fb4d92b59750c6b979fbba, user_id=28b6f8d640e5403f8d50cb4bb9d50448, role_id=df8f9f844b1b439181d75a07c1a93418)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:38.489010 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1012/2013] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 16:00:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 16:00:38.490689 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271e1a3a-8393-4fda-b219-b1f093b5d0c0 admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:38.493633 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4bb17c-581e-406d-a833-ddb21b861b5b None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:38.503852 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4bb17c-581e-406d-a833-ddb21b861b5b None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:38.508397 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1002/2014] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 30 16:00:38 2022] PUT /identity/v3/projects/931345f1d3fb4d92b59750c6b979fbba/users/28b6f8d640e5403f8d50cb4bb9d50448/roles/df8f9f844b1b439181d75a07c1a93418 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 16:00:38.517275 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9372d159-82fa-42e8-8b7d-7f3562bd63c2 None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:38.517848 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9372d159-82fa-42e8-8b7d-7f3562bd63c2 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:38.517848 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9372d159-82fa-42e8-8b7d-7f3562bd63c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:38.546848 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-9372d159-82fa-42e8-8b7d-7f3562bd63c2 None None] MFA Rules not processed for user `28b6f8d640e5403f8d50cb4bb9d50448`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:00:38.548187 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-9372d159-82fa-42e8-8b7d-7f3562bd63c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:00:38.566237 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:00:38.582714 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bbed85-8d34-495a-a6f1-57952e56b85c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:38.584255 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bbed85-8d34-495a-a6f1-57952e56b85c admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:38.584594 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bbed85-8d34-495a-a6f1-57952e56b85c admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:38.585128 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bbed85-8d34-495a-a6f1-57952e56b85c admin admin] PATH_INFO: `/v3/domains` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:38.585909 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17bbed85-8d34-495a-a6f1-57952e56b85c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:00:38.586711 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17bbed85-8d34-495a-a6f1-57952e56b85c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:38.587235 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:00:38.588096 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17bbed85-8d34-495a-a6f1-57952e56b85c admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:38.601514 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1013/2015] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Aug 30 16:00:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 16:00:38.605684 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bbed85-8d34-495a-a6f1-57952e56b85c admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:38.615775 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bbed85-8d34-495a-a6f1-57952e56b85c admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:38.620468 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b33f5d-0a13-48de-a284-85e10b6d5cea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:38.621068 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b33f5d-0a13-48de-a284-85e10b6d5cea admin admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:38.621381 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b33f5d-0a13-48de-a284-85e10b6d5cea admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:38.621652 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b33f5d-0a13-48de-a284-85e10b6d5cea admin admin] PATH_INFO: `/v3/projects` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:38.622292 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b33f5d-0a13-48de-a284-85e10b6d5cea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:38.624224 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b33f5d-0a13-48de-a284-85e10b6d5cea admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:38.641046 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1003/2016] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 16:00:38 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 16:00:38.646246 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9372d159-82fa-42e8-8b7d-7f3562bd63c2 None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:38.659248 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9372d159-82fa-42e8-8b7d-7f3562bd63c2 None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:38.664027 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f723f39d-466e-4c50-8cf0-e45bc4ccdd9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:38.664646 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f723f39d-466e-4c50-8cf0-e45bc4ccdd9b admin admin] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:38.664920 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f723f39d-466e-4c50-8cf0-e45bc4ccdd9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:38.665271 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1014/2017] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Aug 30 16:00:38 2022] POST /identity/v3/projects => generated 365 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 16:00:38.668427 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f723f39d-466e-4c50-8cf0-e45bc4ccdd9b admin admin] PATH_INFO: `/v3/projects` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:38.669091 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f723f39d-466e-4c50-8cf0-e45bc4ccdd9b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:38.669192 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b33f5d-0a13-48de-a284-85e10b6d5cea admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:38.671160 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f723f39d-466e-4c50-8cf0-e45bc4ccdd9b admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:38.702931 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1004/2018] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Aug 30 16:00:38 2022] POST /identity/v3/projects => generated 377 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:00:38.707240 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f723f39d-466e-4c50-8cf0-e45bc4ccdd9b admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:38.734171 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b33f5d-0a13-48de-a284-85e10b6d5cea admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:38.775172 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f723f39d-466e-4c50-8cf0-e45bc4ccdd9b admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:38.780275 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d87c9b4-2097-4211-a3c2-0aa4bebb33df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:38.780898 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d87c9b4-2097-4211-a3c2-0aa4bebb33df admin admin] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:38.781182 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d87c9b4-2097-4211-a3c2-0aa4bebb33df admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:38.781491 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d87c9b4-2097-4211-a3c2-0aa4bebb33df admin admin] PATH_INFO: `/v3/users` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:38.782165 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d87c9b4-2097-4211-a3c2-0aa4bebb33df admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:38.784486 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d87c9b4-2097-4211-a3c2-0aa4bebb33df admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:38.811209 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fa9af4-d7df-4949-b1ff-fbafb9e8fb63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:38.811910 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fa9af4-d7df-4949-b1ff-fbafb9e8fb63 admin admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:38.812187 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fa9af4-d7df-4949-b1ff-fbafb9e8fb63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:38.812452 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fa9af4-d7df-4949-b1ff-fbafb9e8fb63 admin admin] PATH_INFO: `/v3/users` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:38.816083 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1005/2019] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Aug 30 16:00:38 2022] POST /identity/v3/users => generated 295 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 16:00:38.816223 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2fa9af4-d7df-4949-b1ff-fbafb9e8fb63 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:38.816223 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2fa9af4-d7df-4949-b1ff-fbafb9e8fb63 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:38.817058 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d87c9b4-2097-4211-a3c2-0aa4bebb33df admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:38.827236 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d87c9b4-2097-4211-a3c2-0aa4bebb33df admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:38.833349 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a35d67-8414-4984-ba2c-a2ba351ea894 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:38.834368 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a35d67-8414-4984-ba2c-a2ba351ea894 admin admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:38.835041 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a35d67-8414-4984-ba2c-a2ba351ea894 admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:38.835372 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a35d67-8414-4984-ba2c-a2ba351ea894 admin admin] PATH_INFO: `/v3/roles` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:38.836232 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a35d67-8414-4984-ba2c-a2ba351ea894 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:38.838173 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a35d67-8414-4984-ba2c-a2ba351ea894 admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:38.849615 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1015/2020] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Aug 30 16:00:38 2022] POST /identity/v3/users => generated 283 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 16:00:38.850949 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fa9af4-d7df-4949-b1ff-fbafb9e8fb63 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:38.852715 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1006/2021] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 30 16:00:38 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 16:00:38.853619 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a35d67-8414-4984-ba2c-a2ba351ea894 admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:38.865027 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a35d67-8414-4984-ba2c-a2ba351ea894 admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:38.867317 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fa9af4-d7df-4949-b1ff-fbafb9e8fb63 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:38.870515 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db164425-ffd6-4e76-b0d9-ba7a8ec99f21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:38.871194 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db164425-ffd6-4e76-b0d9-ba7a8ec99f21 admin admin] REQUEST_METHOD: `PUT` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:38.871468 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db164425-ffd6-4e76-b0d9-ba7a8ec99f21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:38.871738 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db164425-ffd6-4e76-b0d9-ba7a8ec99f21 admin admin] PATH_INFO: `/v3/projects/fbeca0242d5b46e58a930d67c257c0d1/users/548e1e36f955494cb7e9826aed29c292/roles/f265386950a641ac8413b53eda4ae34d` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:38.874336 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7626712d-dc66-433c-887b-aa2fd16fac91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:38.874960 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7626712d-dc66-433c-887b-aa2fd16fac91 admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:38.875235 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7626712d-dc66-433c-887b-aa2fd16fac91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:38.876305 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7626712d-dc66-433c-887b-aa2fd16fac91 admin admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:38.876305 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7626712d-dc66-433c-887b-aa2fd16fac91 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:38.879076 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7626712d-dc66-433c-887b-aa2fd16fac91 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:38.889806 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db164425-ffd6-4e76-b0d9-ba7a8ec99f21 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fbeca0242d5b46e58a930d67c257c0d1, user_id=548e1e36f955494cb7e9826aed29c292, role_id=f265386950a641ac8413b53eda4ae34d)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:38.894518 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db164425-ffd6-4e76-b0d9-ba7a8ec99f21 admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:38.894681 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1016/2022] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 30 16:00:38 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:00:38.896545 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7626712d-dc66-433c-887b-aa2fd16fac91 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:38.913093 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7626712d-dc66-433c-887b-aa2fd16fac91 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:38.916323 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1007/2023] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 30 16:00:38 2022] PUT /identity/v3/projects/fbeca0242d5b46e58a930d67c257c0d1/users/548e1e36f955494cb7e9826aed29c292/roles/f265386950a641ac8413b53eda4ae34d => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:00:38.920391 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db164425-ffd6-4e76-b0d9-ba7a8ec99f21 admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:38.981289 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:00:39.006844 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7835cad-6f22-4e22-bc88-13bf1b7c8125 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:39.007622 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7835cad-6f22-4e22-bc88-13bf1b7c8125 admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:39.007969 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7835cad-6f22-4e22-bc88-13bf1b7c8125 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:39.008310 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7835cad-6f22-4e22-bc88-13bf1b7c8125 admin admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:39.009039 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7835cad-6f22-4e22-bc88-13bf1b7c8125 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:39.013066 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7835cad-6f22-4e22-bc88-13bf1b7c8125 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:39.022197 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db164425-ffd6-4e76-b0d9-ba7a8ec99f21 admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:39.032531 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419e2083-087b-4f51-9953-a993bc279591 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:39.035073 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419e2083-087b-4f51-9953-a993bc279591 admin admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:39.035584 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419e2083-087b-4f51-9953-a993bc279591 admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:39.036174 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1017/2024] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 30 16:00:38 2022] GET /identity/v3/roles => generated 1420 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:00:39.036881 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419e2083-087b-4f51-9953-a993bc279591 admin admin] PATH_INFO: `/v3/roles` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:39.038426 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419e2083-087b-4f51-9953-a993bc279591 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:39.040133 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7835cad-6f22-4e22-bc88-13bf1b7c8125 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:39.043392 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419e2083-087b-4f51-9953-a993bc279591 admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:39.051006 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:00:39.063763 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7835cad-6f22-4e22-bc88-13bf1b7c8125 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:39.072453 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1008/2025] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 30 16:00:38 2022] GET /identity/v3/roles => generated 1420 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:00:39.074888 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419e2083-087b-4f51-9953-a993bc279591 admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:39.075834 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31471644-6781-461b-85b1-bd4b36a84ce3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:39.076610 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31471644-6781-461b-85b1-bd4b36a84ce3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:39.076967 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31471644-6781-461b-85b1-bd4b36a84ce3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:39.077333 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31471644-6781-461b-85b1-bd4b36a84ce3 admin admin] PATH_INFO: `/v3/projects/07444b32020147ef8da5cdffd8591021/users/e2b50ec1433f44038e66b163c2c9ccb6/roles/df8f9f844b1b439181d75a07c1a93418` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:39.098109 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419e2083-087b-4f51-9953-a993bc279591 admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:39.104376 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31471644-6781-461b-85b1-bd4b36a84ce3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=07444b32020147ef8da5cdffd8591021, user_id=e2b50ec1433f44038e66b163c2c9ccb6, role_id=df8f9f844b1b439181d75a07c1a93418)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:39.107223 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31471644-6781-461b-85b1-bd4b36a84ce3 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:39.113391 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc504680-03c3-4e79-91f9-ddaa1a7eff03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:39.114329 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc504680-03c3-4e79-91f9-ddaa1a7eff03 admin admin] REQUEST_METHOD: `PUT` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:39.114772 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc504680-03c3-4e79-91f9-ddaa1a7eff03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:39.114992 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc504680-03c3-4e79-91f9-ddaa1a7eff03 admin admin] PATH_INFO: `/v3/projects/fbeca0242d5b46e58a930d67c257c0d1/users/548e1e36f955494cb7e9826aed29c292/roles/f265386950a641ac8413b53eda4ae34d` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:39.118636 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc504680-03c3-4e79-91f9-ddaa1a7eff03 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fbeca0242d5b46e58a930d67c257c0d1, user_id=548e1e36f955494cb7e9826aed29c292, role_id=f265386950a641ac8413b53eda4ae34d)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:39.121696 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc504680-03c3-4e79-91f9-ddaa1a7eff03 admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:39.133297 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1018/2026] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 30 16:00:39 2022] PUT /identity/v3/projects/07444b32020147ef8da5cdffd8591021/users/e2b50ec1433f44038e66b163c2c9ccb6/roles/df8f9f844b1b439181d75a07c1a93418 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:00:39.141960 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1009/2027] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 30 16:00:39 2022] PUT /identity/v3/projects/fbeca0242d5b46e58a930d67c257c0d1/users/548e1e36f955494cb7e9826aed29c292/roles/f265386950a641ac8413b53eda4ae34d => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 16:00:39.143781 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de38fd51-8ba0-4c22-8685-46efaeb66b4c None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:39.144096 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de38fd51-8ba0-4c22-8685-46efaeb66b4c None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:39.144391 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de38fd51-8ba0-4c22-8685-46efaeb66b4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:39.153597 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e68d5d-7840-40cb-b8a9-262b944e6785 None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:39.153884 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e68d5d-7840-40cb-b8a9-262b944e6785 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:39.154161 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e68d5d-7840-40cb-b8a9-262b944e6785 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:39.180064 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-de38fd51-8ba0-4c22-8685-46efaeb66b4c None None] MFA Rules not processed for user `e2b50ec1433f44038e66b163c2c9ccb6`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:00:39.181334 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-de38fd51-8ba0-4c22-8685-46efaeb66b4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:00:39.227238 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-75e68d5d-7840-40cb-b8a9-262b944e6785 None None] MFA Rules not processed for user `548e1e36f955494cb7e9826aed29c292`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:00:39.230506 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-75e68d5d-7840-40cb-b8a9-262b944e6785 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:00:39.303247 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1019/2028] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 16:00:39 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:00:39.307436 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de38fd51-8ba0-4c22-8685-46efaeb66b4c None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:39.324713 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1010/2029] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 16:00:39 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:00:39.337698 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e68d5d-7840-40cb-b8a9-262b944e6785 None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:39.339463 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-75e68d5d-7840-40cb-b8a9-262b944e6785 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:00:39.384183 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de38fd51-8ba0-4c22-8685-46efaeb66b4c None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:39.389130 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6261ab6-b2c7-4b3d-802f-475da7d59ff3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:39.389781 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6261ab6-b2c7-4b3d-802f-475da7d59ff3 admin admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:39.390057 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6261ab6-b2c7-4b3d-802f-475da7d59ff3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:39.390321 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6261ab6-b2c7-4b3d-802f-475da7d59ff3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:39.390962 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6261ab6-b2c7-4b3d-802f-475da7d59ff3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:39.394160 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6261ab6-b2c7-4b3d-802f-475da7d59ff3 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:39.409839 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e68d5d-7840-40cb-b8a9-262b944e6785 None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:39.415155 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26b5ad0-5295-4704-b6e0-c07feda18d56 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0100a04821804993866ea20f159663f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:39.415927 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26b5ad0-5295-4704-b6e0-c07feda18d56 service cinder] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:39.416284 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26b5ad0-5295-4704-b6e0-c07feda18d56 service cinder] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:39.416658 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26b5ad0-5295-4704-b6e0-c07feda18d56 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:39.431555 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26b5ad0-5295-4704-b6e0-c07feda18d56 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:39.434114 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26b5ad0-5295-4704-b6e0-c07feda18d56 service cinder] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:39.439236 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1020/2030] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Aug 30 16:00:39 2022] POST /identity/v3/projects => generated 363 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 16:00:39.443831 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6261ab6-b2c7-4b3d-802f-475da7d59ff3 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:39.465998 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1011/2031] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 16:00:39 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 16:00:39.504329 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:00:39.532695 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6261ab6-b2c7-4b3d-802f-475da7d59ff3 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:39.533314 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26b5ad0-5295-4704-b6e0-c07feda18d56 service cinder] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:39.534982 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-d26b5ad0-5295-4704-b6e0-c07feda18d56 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:00:39.538390 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5f275b-acd7-49c4-89e9-e2bb655a63b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:39.539391 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5f275b-acd7-49c4-89e9-e2bb655a63b4 admin admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:39.539524 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5f275b-acd7-49c4-89e9-e2bb655a63b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:39.539692 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5f275b-acd7-49c4-89e9-e2bb655a63b4 admin admin] PATH_INFO: `/v3/users` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:39.540323 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5f275b-acd7-49c4-89e9-e2bb655a63b4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:39.542295 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5f275b-acd7-49c4-89e9-e2bb655a63b4 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:39.547088 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26b5ad0-5295-4704-b6e0-c07feda18d56 service cinder] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:39.588064 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1021/2032] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Aug 30 16:00:39 2022] POST /identity/v3/users => generated 288 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:00:39.592049 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5f275b-acd7-49c4-89e9-e2bb655a63b4 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:39.603395 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5f275b-acd7-49c4-89e9-e2bb655a63b4 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:39.608812 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3ee079-1c59-4e68-9fae-75a87debb873 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:39.618327 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3ee079-1c59-4e68-9fae-75a87debb873 admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:39.621468 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3ee079-1c59-4e68-9fae-75a87debb873 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:39.621838 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3ee079-1c59-4e68-9fae-75a87debb873 admin admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:39.622623 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3ee079-1c59-4e68-9fae-75a87debb873 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:39.624345 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3ee079-1c59-4e68-9fae-75a87debb873 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:39.653341 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1022/2033] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 30 16:00:39 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:00:39.654416 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3ee079-1c59-4e68-9fae-75a87debb873 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:39.667255 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbd1d6b-df16-4c32-987a-d02276cd23f0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0100a04821804993866ea20f159663f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:39.669291 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbd1d6b-df16-4c32-987a-d02276cd23f0 service cinder] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:39.669656 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbd1d6b-df16-4c32-987a-d02276cd23f0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:39.670003 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbd1d6b-df16-4c32-987a-d02276cd23f0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:39.673403 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-cdbd1d6b-df16-4c32-987a-d02276cd23f0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:00:39.677432 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3ee079-1c59-4e68-9fae-75a87debb873 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:39.688017 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c11524a-3a39-4d4e-9fc9-67b786bfe8ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:39.688769 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c11524a-3a39-4d4e-9fc9-67b786bfe8ec admin admin] REQUEST_METHOD: `PUT` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:39.689056 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c11524a-3a39-4d4e-9fc9-67b786bfe8ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:39.689476 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c11524a-3a39-4d4e-9fc9-67b786bfe8ec admin admin] PATH_INFO: `/v3/projects/06b40aecb9054e17abd7577c309d9d6a/users/c400363e29a74a419c5f56fb599e38a2/roles/f265386950a641ac8413b53eda4ae34d` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:39.705597 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:00:39.714942 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c11524a-3a39-4d4e-9fc9-67b786bfe8ec admin admin] RBAC: Authorizing `identity:create_grant(project_id=06b40aecb9054e17abd7577c309d9d6a, user_id=c400363e29a74a419c5f56fb599e38a2, role_id=f265386950a641ac8413b53eda4ae34d)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:39.717799 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c11524a-3a39-4d4e-9fc9-67b786bfe8ec admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:39.735570 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1023/2034] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 30 16:00:39 2022] PUT /identity/v3/projects/06b40aecb9054e17abd7577c309d9d6a/users/c400363e29a74a419c5f56fb599e38a2/roles/f265386950a641ac8413b53eda4ae34d => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 16:00:39.739911 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c11524a-3a39-4d4e-9fc9-67b786bfe8ec admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:39.752792 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdbd1d6b-df16-4c32-987a-d02276cd23f0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:39.754649 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdbd1d6b-df16-4c32-987a-d02276cd23f0 service cinder] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:39.770124 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1012/2035] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 16:00:39 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 237 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 16:00:39.831319 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c11524a-3a39-4d4e-9fc9-67b786bfe8ec admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:39.839485 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e19fd68-d6be-46ae-ad5f-01fed0f50172 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:39.840291 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e19fd68-d6be-46ae-ad5f-01fed0f50172 admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:39.840573 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e19fd68-d6be-46ae-ad5f-01fed0f50172 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:39.840951 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e19fd68-d6be-46ae-ad5f-01fed0f50172 admin admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:39.841711 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e19fd68-d6be-46ae-ad5f-01fed0f50172 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:39.843463 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e19fd68-d6be-46ae-ad5f-01fed0f50172 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:39.861556 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1024/2036] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 30 16:00:39 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:00:39.862590 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbd1d6b-df16-4c32-987a-d02276cd23f0 service cinder] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:39.875676 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbd1d6b-df16-4c32-987a-d02276cd23f0 service cinder] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:39.882764 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c7336b-949d-4f21-8837-0c3660cbea36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:39.886281 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c7336b-949d-4f21-8837-0c3660cbea36 admin admin] REQUEST_METHOD: `PUT` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:39.886281 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c7336b-949d-4f21-8837-0c3660cbea36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:39.886281 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c7336b-949d-4f21-8837-0c3660cbea36 admin admin] PATH_INFO: `/v3/projects/06b40aecb9054e17abd7577c309d9d6a/users/c400363e29a74a419c5f56fb599e38a2/roles/f265386950a641ac8413b53eda4ae34d` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:39.886770 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c7336b-949d-4f21-8837-0c3660cbea36 admin admin] RBAC: Authorizing `identity:create_grant(project_id=06b40aecb9054e17abd7577c309d9d6a, user_id=c400363e29a74a419c5f56fb599e38a2, role_id=f265386950a641ac8413b53eda4ae34d)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:39.889815 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c7336b-949d-4f21-8837-0c3660cbea36 admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:39.914264 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1013/2037] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 30 16:00:39 2022] PUT /identity/v3/projects/06b40aecb9054e17abd7577c309d9d6a/users/c400363e29a74a419c5f56fb599e38a2/roles/f265386950a641ac8413b53eda4ae34d => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 16:00:39.916637 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35f7cc2-eda2-47d3-abd9-562b16858eaf None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:39.916925 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35f7cc2-eda2-47d3-abd9-562b16858eaf None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:39.917199 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35f7cc2-eda2-47d3-abd9-562b16858eaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:39.955853 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-b35f7cc2-eda2-47d3-abd9-562b16858eaf None None] MFA Rules not processed for user `c400363e29a74a419c5f56fb599e38a2`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:00:39.957388 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-b35f7cc2-eda2-47d3-abd9-562b16858eaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:00:40.081462 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1025/2038] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 16:00:39 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:00:40.083512 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c7336b-949d-4f21-8837-0c3660cbea36 admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:40.184206 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c7336b-949d-4f21-8837-0c3660cbea36 admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:40.197887 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a010fb81-9312-43a1-8679-9e05f376ae1a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0100a04821804993866ea20f159663f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:40.198810 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a010fb81-9312-43a1-8679-9e05f376ae1a service cinder] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:40.199108 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a010fb81-9312-43a1-8679-9e05f376ae1a service cinder] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:40.199392 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a010fb81-9312-43a1-8679-9e05f376ae1a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:40.210945 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a010fb81-9312-43a1-8679-9e05f376ae1a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:40.212745 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a010fb81-9312-43a1-8679-9e05f376ae1a service cinder] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:40.224722 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1014/2039] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 16:00:40 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:00:40.830338 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35f7cc2-eda2-47d3-abd9-562b16858eaf None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:40.850753 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:00:40.946351 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35f7cc2-eda2-47d3-abd9-562b16858eaf None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:40.958762 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e07707-2b8e-4179-b19b-110b4b8376e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:40.959457 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e07707-2b8e-4179-b19b-110b4b8376e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:40.960513 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e07707-2b8e-4179-b19b-110b4b8376e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:40.969499 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e07707-2b8e-4179-b19b-110b4b8376e0 admin admin] PATH_INFO: `/v3/users/e2b50ec1433f44038e66b163c2c9ccb6` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:40.971011 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e07707-2b8e-4179-b19b-110b4b8376e0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e2b50ec1433f44038e66b163c2c9ccb6)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:40.973094 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e07707-2b8e-4179-b19b-110b4b8376e0 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:41.060890 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-81e07707-2b8e-4179-b19b-110b4b8376e0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e2b50ec1433f44038e66b163c2c9ccb6'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:41.079610 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-81e07707-2b8e-4179-b19b-110b4b8376e0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e2b50ec1433f44038e66b163c2c9ccb6'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:41.092269 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-81e07707-2b8e-4179-b19b-110b4b8376e0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e2b50ec1433f44038e66b163c2c9ccb6'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:41.092964 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-81e07707-2b8e-4179-b19b-110b4b8376e0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e2b50ec1433f44038e66b163c2c9ccb6'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:41.101266 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-81e07707-2b8e-4179-b19b-110b4b8376e0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e2b50ec1433f44038e66b163c2c9ccb6'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:41.107160 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1026/2040] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Aug 30 16:00:40 2022] DELETE /identity/v3/users/e2b50ec1433f44038e66b163c2c9ccb6 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:00:41.111735 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a405576d-d3a2-4063-bc8a-69f946261226 None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:41.112233 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a405576d-d3a2-4063-bc8a-69f946261226 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:41.112339 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a405576d-d3a2-4063-bc8a-69f946261226 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:41.135315 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-a405576d-d3a2-4063-bc8a-69f946261226 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:00:41.136574 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-a405576d-d3a2-4063-bc8a-69f946261226 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:00:41.213357 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1015/2041] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 16:00:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:00:41.223207 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e07707-2b8e-4179-b19b-110b4b8376e0 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:41.224811 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-81e07707-2b8e-4179-b19b-110b4b8376e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:00:41.304033 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e07707-2b8e-4179-b19b-110b4b8376e0 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:41.308682 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35209e9b-9611-41d2-a88d-d412854c7ae2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f4157105f3488f807b7475008ab722', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:41.309366 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35209e9b-9611-41d2-a88d-d412854c7ae2 service neutron] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:41.309675 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35209e9b-9611-41d2-a88d-d412854c7ae2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:41.309944 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35209e9b-9611-41d2-a88d-d412854c7ae2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:41.320675 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35209e9b-9611-41d2-a88d-d412854c7ae2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:41.322196 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35209e9b-9611-41d2-a88d-d412854c7ae2 service neutron] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:41.332937 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1027/2042] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 16:00:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:00:41.952245 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a405576d-d3a2-4063-bc8a-69f946261226 None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:41.953832 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-a405576d-d3a2-4063-bc8a-69f946261226 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:00:41.966527 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a405576d-d3a2-4063-bc8a-69f946261226 None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:41.971909 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f229274-1102-4474-8245-d3036cbe1b21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:41.972560 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f229274-1102-4474-8245-d3036cbe1b21 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:41.972829 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f229274-1102-4474-8245-d3036cbe1b21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:41.973147 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f229274-1102-4474-8245-d3036cbe1b21 admin admin] PATH_INFO: `/v3/projects/07444b32020147ef8da5cdffd8591021` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:41.974289 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f229274-1102-4474-8245-d3036cbe1b21 admin admin] RBAC: Authorizing `identity:delete_project(project_id=07444b32020147ef8da5cdffd8591021)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:41.976170 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f229274-1102-4474-8245-d3036cbe1b21 admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:42.051968 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-8f229274-1102-4474-8245-d3036cbe1b21 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '07444b32020147ef8da5cdffd8591021', 'request_id': 'req-8f229274-1102-4474-8245-d3036cbe1b21'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:42.054586 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=95,keystone:INSERT=1,keystone:DELETE=4 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:00:42.069640 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-8f229274-1102-4474-8245-d3036cbe1b21 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '07444b32020147ef8da5cdffd8591021', 'request_id': 'req-8f229274-1102-4474-8245-d3036cbe1b21'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:42.081483 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-8f229274-1102-4474-8245-d3036cbe1b21 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '07444b32020147ef8da5cdffd8591021', 'request_id': 'req-8f229274-1102-4474-8245-d3036cbe1b21'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:42.098040 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-8f229274-1102-4474-8245-d3036cbe1b21 admin admin] The token cache is being invalidate because project 07444b32020147ef8da5cdffd8591021 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81512) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 16:00:42.098399 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-8f229274-1102-4474-8245-d3036cbe1b21 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:42.099054 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-8f229274-1102-4474-8245-d3036cbe1b21 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:42.100951 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1016/2043] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Aug 30 16:00:41 2022] DELETE /identity/v3/projects/07444b32020147ef8da5cdffd8591021 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:00:42.105106 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35209e9b-9611-41d2-a88d-d412854c7ae2 service neutron] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:42.115302 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35209e9b-9611-41d2-a88d-d412854c7ae2 service neutron] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:42.155272 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:00:42.194531 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd74d065-3165-41a3-a724-31673c3ac596 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:42.195395 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd74d065-3165-41a3-a724-31673c3ac596 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:42.195593 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd74d065-3165-41a3-a724-31673c3ac596 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:42.196046 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd74d065-3165-41a3-a724-31673c3ac596 admin admin] PATH_INFO: `/v3/users/c400363e29a74a419c5f56fb599e38a2` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:42.197296 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd74d065-3165-41a3-a724-31673c3ac596 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c400363e29a74a419c5f56fb599e38a2)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:42.199236 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd74d065-3165-41a3-a724-31673c3ac596 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:42.260171 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-dd74d065-3165-41a3-a724-31673c3ac596 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c400363e29a74a419c5f56fb599e38a2'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:42.273065 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-dd74d065-3165-41a3-a724-31673c3ac596 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c400363e29a74a419c5f56fb599e38a2'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:42.283602 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-dd74d065-3165-41a3-a724-31673c3ac596 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c400363e29a74a419c5f56fb599e38a2'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:42.287239 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-dd74d065-3165-41a3-a724-31673c3ac596 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c400363e29a74a419c5f56fb599e38a2'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:42.296933 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-dd74d065-3165-41a3-a724-31673c3ac596 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c400363e29a74a419c5f56fb599e38a2'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:42.303876 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1028/2044] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Aug 30 16:00:42 2022] DELETE /identity/v3/users/c400363e29a74a419c5f56fb599e38a2 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:00:42.603725 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f229274-1102-4474-8245-d3036cbe1b21 admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:42.605473 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-8f229274-1102-4474-8245-d3036cbe1b21 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:00:42.627817 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f229274-1102-4474-8245-d3036cbe1b21 admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:42.651265 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82408bc-8c24-4ce2-a929-24c2dcb7e4a8 None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:42.651265 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82408bc-8c24-4ce2-a929-24c2dcb7e4a8 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:42.651640 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82408bc-8c24-4ce2-a929-24c2dcb7e4a8 None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:42.653814 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1029/2045] 10.222.0.20 () {54 vars in 921 bytes} [Tue Aug 30 16:00:42 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 16:00:42.697442 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c5b390-0ae3-42a6-b113-d4fa0f09aa99 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:42.697742 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c5b390-0ae3-42a6-b113-d4fa0f09aa99 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:42.698075 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c5b390-0ae3-42a6-b113-d4fa0f09aa99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:42.732259 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-59c5b390-0ae3-42a6-b113-d4fa0f09aa99 None None] MFA Rules not processed for user `bc12c6d31eab4a17a160250adc2380c8`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:00:42.734239 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7527c65b-adff-487d-970e-bcc1b45749c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:42.735081 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-59c5b390-0ae3-42a6-b113-d4fa0f09aa99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:00:42.735206 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7527c65b-adff-487d-970e-bcc1b45749c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:42.735441 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7527c65b-adff-487d-970e-bcc1b45749c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:42.736785 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7527c65b-adff-487d-970e-bcc1b45749c0 admin admin] PATH_INFO: `/v3/projects/06b40aecb9054e17abd7577c309d9d6a` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:42.738903 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7527c65b-adff-487d-970e-bcc1b45749c0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=06b40aecb9054e17abd7577c309d9d6a)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:42.741261 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7527c65b-adff-487d-970e-bcc1b45749c0 admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:42.847804 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1030/2046] 10.222.0.20 () {58 vars in 1018 bytes} [Tue Aug 30 16:00:42 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:00:42.849009 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c5b390-0ae3-42a6-b113-d4fa0f09aa99 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:42.851926 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-59c5b390-0ae3-42a6-b113-d4fa0f09aa99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:00:42.856785 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-7527c65b-adff-487d-970e-bcc1b45749c0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '06b40aecb9054e17abd7577c309d9d6a', 'request_id': 'req-7527c65b-adff-487d-970e-bcc1b45749c0'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:42.870550 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c5b390-0ae3-42a6-b113-d4fa0f09aa99 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:42.873168 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-7527c65b-adff-487d-970e-bcc1b45749c0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '06b40aecb9054e17abd7577c309d9d6a', 'request_id': 'req-7527c65b-adff-487d-970e-bcc1b45749c0'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:42.884502 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-7527c65b-adff-487d-970e-bcc1b45749c0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '06b40aecb9054e17abd7577c309d9d6a', 'request_id': 'req-7527c65b-adff-487d-970e-bcc1b45749c0'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:42.898711 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-7527c65b-adff-487d-970e-bcc1b45749c0 admin admin] The token cache is being invalidate because project 06b40aecb9054e17abd7577c309d9d6a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81512) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 16:00:42.898955 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-7527c65b-adff-487d-970e-bcc1b45749c0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:42.899780 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-7527c65b-adff-487d-970e-bcc1b45749c0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:42.901727 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1017/2047] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Aug 30 16:00:42 2022] DELETE /identity/v3/projects/06b40aecb9054e17abd7577c309d9d6a => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:00:42.907248 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5ac1e8-edae-4d70-9baa-707e532c0249 None None] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:42.907248 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5ac1e8-edae-4d70-9baa-707e532c0249 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:42.907248 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5ac1e8-edae-4d70-9baa-707e532c0249 None None] PATH_INFO: `/` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:42.908740 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1018/2048] 10.222.0.20 () {54 vars in 961 bytes} [Tue Aug 30 16:00:42 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 16:00:42.910231 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8127aed1-7514-47dd-9721-e51fa4cfbe63 None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:42.910592 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8127aed1-7514-47dd-9721-e51fa4cfbe63 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:42.910925 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8127aed1-7514-47dd-9721-e51fa4cfbe63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:42.926619 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:00:42.931951 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-8127aed1-7514-47dd-9721-e51fa4cfbe63 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:00:42.934311 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-8127aed1-7514-47dd-9721-e51fa4cfbe63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:00:42.942456 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeed4fbb-5610-47bb-b38c-c8259523a88e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:42.943164 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeed4fbb-5610-47bb-b38c-c8259523a88e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:42.943511 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeed4fbb-5610-47bb-b38c-c8259523a88e admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:42.943845 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeed4fbb-5610-47bb-b38c-c8259523a88e admin admin] PATH_INFO: `/v3/users/096f60ad096243d3829d256db17a5f8a` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:42.945052 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeed4fbb-5610-47bb-b38c-c8259523a88e admin admin] RBAC: Authorizing `identity:delete_user(user_id=096f60ad096243d3829d256db17a5f8a)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:42.947402 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeed4fbb-5610-47bb-b38c-c8259523a88e admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:43.015826 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1019/2049] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 16:00:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:00:43.016500 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-eeed4fbb-5610-47bb-b38c-c8259523a88e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '096f60ad096243d3829d256db17a5f8a'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:43.017444 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e79ba1-d3b0-4891-86bf-ee44a7aa1d5c None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:43.017872 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e79ba1-d3b0-4891-86bf-ee44a7aa1d5c None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:43.018253 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e79ba1-d3b0-4891-86bf-ee44a7aa1d5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:43.044097 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-eeed4fbb-5610-47bb-b38c-c8259523a88e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '096f60ad096243d3829d256db17a5f8a'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:43.045958 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-95e79ba1-d3b0-4891-86bf-ee44a7aa1d5c None None] MFA Rules not processed for user `bc12c6d31eab4a17a160250adc2380c8`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:00:43.045958 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-95e79ba1-d3b0-4891-86bf-ee44a7aa1d5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:00:43.051089 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:INSERT=2,keystone:DELETE=10,keystone:SELECT=88 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:00:43.058398 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-eeed4fbb-5610-47bb-b38c-c8259523a88e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '096f60ad096243d3829d256db17a5f8a'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:43.059280 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-eeed4fbb-5610-47bb-b38c-c8259523a88e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '096f60ad096243d3829d256db17a5f8a'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:43.070159 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-eeed4fbb-5610-47bb-b38c-c8259523a88e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '096f60ad096243d3829d256db17a5f8a'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:43.079052 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1031/2050] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Aug 30 16:00:42 2022] DELETE /identity/v3/users/096f60ad096243d3829d256db17a5f8a => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:00:43.080610 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480ab144-1ab6-493c-8df3-e8ea416884a5 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:43.081001 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480ab144-1ab6-493c-8df3-e8ea416884a5 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:43.082063 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480ab144-1ab6-493c-8df3-e8ea416884a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:43.104243 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-480ab144-1ab6-493c-8df3-e8ea416884a5 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:00:43.105666 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-480ab144-1ab6-493c-8df3-e8ea416884a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:00:43.126486 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1020/2051] 10.222.0.20 () {58 vars in 1058 bytes} [Tue Aug 30 16:00:43 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:00:43.127288 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e79ba1-d3b0-4891-86bf-ee44a7aa1d5c None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:43.129344 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-95e79ba1-d3b0-4891-86bf-ee44a7aa1d5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:00:43.146265 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e79ba1-d3b0-4891-86bf-ee44a7aa1d5c None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:43.192255 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1032/2052] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 16:00:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:00:43.195578 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480ab144-1ab6-493c-8df3-e8ea416884a5 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:43.221910 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484eb512-b0c8-410c-8e7a-8be8902fe45d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:43.222472 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484eb512-b0c8-410c-8e7a-8be8902fe45d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:43.223057 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484eb512-b0c8-410c-8e7a-8be8902fe45d admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:43.223057 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484eb512-b0c8-410c-8e7a-8be8902fe45d admin admin] PATH_INFO: `/v3/users/c1243529bd0347bdb252197a225d679e` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:43.224594 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484eb512-b0c8-410c-8e7a-8be8902fe45d admin admin] RBAC: Authorizing `identity:delete_user(user_id=c1243529bd0347bdb252197a225d679e)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:43.227335 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484eb512-b0c8-410c-8e7a-8be8902fe45d admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:43.277528 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480ab144-1ab6-493c-8df3-e8ea416884a5 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:43.283365 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-484eb512-b0c8-410c-8e7a-8be8902fe45d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c1243529bd0347bdb252197a225d679e'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:43.284273 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16c5203-fd2a-41d6-989b-0e5ec8b534de service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc12c6d31eab4a17a160250adc2380c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:43.285118 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16c5203-fd2a-41d6-989b-0e5ec8b534de service placement] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:43.285483 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16c5203-fd2a-41d6-989b-0e5ec8b534de service placement] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:43.286135 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16c5203-fd2a-41d6-989b-0e5ec8b534de service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:43.288494 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-c16c5203-fd2a-41d6-989b-0e5ec8b534de service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:00:43.294621 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-484eb512-b0c8-410c-8e7a-8be8902fe45d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c1243529bd0347bdb252197a225d679e'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:43.305986 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16c5203-fd2a-41d6-989b-0e5ec8b534de service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:43.308153 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16c5203-fd2a-41d6-989b-0e5ec8b534de service placement] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:43.316108 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-484eb512-b0c8-410c-8e7a-8be8902fe45d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c1243529bd0347bdb252197a225d679e'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:43.316229 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:00:43.324677 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-484eb512-b0c8-410c-8e7a-8be8902fe45d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c1243529bd0347bdb252197a225d679e'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:43.325694 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-484eb512-b0c8-410c-8e7a-8be8902fe45d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c1243529bd0347bdb252197a225d679e'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:43.329900 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1033/2053] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 16:00:43 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:00:43.331591 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16c5203-fd2a-41d6-989b-0e5ec8b534de service placement] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:43.336155 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1021/2054] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Aug 30 16:00:43 2022] DELETE /identity/v3/users/c1243529bd0347bdb252197a225d679e => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:00:43.337068 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484eb512-b0c8-410c-8e7a-8be8902fe45d admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:43.338657 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-484eb512-b0c8-410c-8e7a-8be8902fe45d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:00:43.350654 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16c5203-fd2a-41d6-989b-0e5ec8b534de service placement] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:43.370052 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:00:43.418793 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5debb7f9-e949-45c1-88e6-b93fc79b5f56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:43.419206 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484eb512-b0c8-410c-8e7a-8be8902fe45d admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:43.419398 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5debb7f9-e949-45c1-88e6-b93fc79b5f56 admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:43.419667 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5debb7f9-e949-45c1-88e6-b93fc79b5f56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:43.419928 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5debb7f9-e949-45c1-88e6-b93fc79b5f56 admin admin] PATH_INFO: `/v3/domains` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:43.420570 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5debb7f9-e949-45c1-88e6-b93fc79b5f56 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:00:43.420864 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5debb7f9-e949-45c1-88e6-b93fc79b5f56 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:43.422591 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5debb7f9-e949-45c1-88e6-b93fc79b5f56 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:43.423967 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0249527-c37d-40cb-b812-7bd65e97908e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:43.424603 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0249527-c37d-40cb-b812-7bd65e97908e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:43.424884 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0249527-c37d-40cb-b812-7bd65e97908e admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:43.425153 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0249527-c37d-40cb-b812-7bd65e97908e admin admin] PATH_INFO: `/v3/projects/972006ddc9c04f1b838860912bbabe0c` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:43.426809 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0249527-c37d-40cb-b812-7bd65e97908e admin admin] RBAC: Authorizing `identity:delete_project(project_id=972006ddc9c04f1b838860912bbabe0c)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:43.428751 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0249527-c37d-40cb-b812-7bd65e97908e admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:43.433980 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1034/2055] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Aug 30 16:00:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:00:43.435213 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bee45ce-e0c6-4816-811b-6f8a89947973 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:43.435501 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bee45ce-e0c6-4816-811b-6f8a89947973 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:43.435798 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bee45ce-e0c6-4816-811b-6f8a89947973 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:43.455122 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-4bee45ce-e0c6-4816-811b-6f8a89947973 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:00:43.456309 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-4bee45ce-e0c6-4816-811b-6f8a89947973 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:00:43.464738 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1035/2056] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 16:00:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:00:43.465524 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bee45ce-e0c6-4816-811b-6f8a89947973 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:43.466745 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-4bee45ce-e0c6-4816-811b-6f8a89947973 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:00:43.478368 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bee45ce-e0c6-4816-811b-6f8a89947973 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:43.480815 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-c0249527-c37d-40cb-b812-7bd65e97908e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '972006ddc9c04f1b838860912bbabe0c', 'request_id': 'req-c0249527-c37d-40cb-b812-7bd65e97908e'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:43.490939 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-c0249527-c37d-40cb-b812-7bd65e97908e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '972006ddc9c04f1b838860912bbabe0c', 'request_id': 'req-c0249527-c37d-40cb-b812-7bd65e97908e'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:43.498173 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-c0249527-c37d-40cb-b812-7bd65e97908e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '972006ddc9c04f1b838860912bbabe0c', 'request_id': 'req-c0249527-c37d-40cb-b812-7bd65e97908e'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:43.509326 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-c0249527-c37d-40cb-b812-7bd65e97908e admin admin] The token cache is being invalidate because project 972006ddc9c04f1b838860912bbabe0c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81512) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 16:00:43.509615 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-c0249527-c37d-40cb-b812-7bd65e97908e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:43.510479 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-c0249527-c37d-40cb-b812-7bd65e97908e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:43.512206 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1022/2057] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Aug 30 16:00:43 2022] DELETE /identity/v3/projects/972006ddc9c04f1b838860912bbabe0c => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:00:43.513050 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0249527-c37d-40cb-b812-7bd65e97908e admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:43.514514 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-c0249527-c37d-40cb-b812-7bd65e97908e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:00:43.536590 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57d5064-c3a2-4a2d-aa35-2ece43407261 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:43.537203 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57d5064-c3a2-4a2d-aa35-2ece43407261 admin admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:43.537502 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57d5064-c3a2-4a2d-aa35-2ece43407261 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:43.537767 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57d5064-c3a2-4a2d-aa35-2ece43407261 admin admin] PATH_INFO: `/v3/projects` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:43.538511 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57d5064-c3a2-4a2d-aa35-2ece43407261 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:43.540413 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57d5064-c3a2-4a2d-aa35-2ece43407261 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:43.573825 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1036/2058] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Aug 30 16:00:43 2022] POST /identity/v3/projects => generated 369 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:00:43.573825 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816b9fdb-e192-4246-a0dc-de0394d7f2bc None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:43.573825 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816b9fdb-e192-4246-a0dc-de0394d7f2bc None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:43.573825 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816b9fdb-e192-4246-a0dc-de0394d7f2bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:43.574286 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0249527-c37d-40cb-b812-7bd65e97908e admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:43.591670 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-816b9fdb-e192-4246-a0dc-de0394d7f2bc None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:00:43.592865 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-816b9fdb-e192-4246-a0dc-de0394d7f2bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:00:43.620833 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb4712f-791e-4237-9007-b337de86e282 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f4157105f3488f807b7475008ab722', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:43.621509 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb4712f-791e-4237-9007-b337de86e282 service neutron] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:43.621742 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb4712f-791e-4237-9007-b337de86e282 service neutron] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:43.622008 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb4712f-791e-4237-9007-b337de86e282 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:43.624081 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-0eb4712f-791e-4237-9007-b337de86e282 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:00:43.653428 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1037/2059] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 16:00:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:00:43.654328 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816b9fdb-e192-4246-a0dc-de0394d7f2bc None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:43.655775 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-816b9fdb-e192-4246-a0dc-de0394d7f2bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:00:43.657901 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:00:43.668507 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816b9fdb-e192-4246-a0dc-de0394d7f2bc None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:43.673397 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0ef9d0-a90e-4f34-9447-bd6a7ffcb37b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:43.674007 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0ef9d0-a90e-4f34-9447-bd6a7ffcb37b admin admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:43.674286 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0ef9d0-a90e-4f34-9447-bd6a7ffcb37b admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:43.674556 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0ef9d0-a90e-4f34-9447-bd6a7ffcb37b admin admin] PATH_INFO: `/v3/users` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:43.675193 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0ef9d0-a90e-4f34-9447-bd6a7ffcb37b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:43.677177 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0ef9d0-a90e-4f34-9447-bd6a7ffcb37b admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:43.687794 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:00:43.700315 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb4712f-791e-4237-9007-b337de86e282 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:43.704626 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1038/2060] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Aug 30 16:00:43 2022] POST /identity/v3/users => generated 285 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:00:43.704774 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb4712f-791e-4237-9007-b337de86e282 service neutron] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:43.704864 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5113dc5d-a7e3-4d55-93c7-c57137776dd6 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:43.704864 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5113dc5d-a7e3-4d55-93c7-c57137776dd6 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:43.704864 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5113dc5d-a7e3-4d55-93c7-c57137776dd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:43.711766 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1023/2061] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 16:00:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:00:43.712538 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb4712f-791e-4237-9007-b337de86e282 service neutron] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:43.721615 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb4712f-791e-4237-9007-b337de86e282 service neutron] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:43.724751 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-5113dc5d-a7e3-4d55-93c7-c57137776dd6 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:00:43.726000 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-5113dc5d-a7e3-4d55-93c7-c57137776dd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:00:43.726820 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcb53e1-a58b-4c20-9ded-d2d7f6164a41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:43.727427 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcb53e1-a58b-4c20-9ded-d2d7f6164a41 admin admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:43.728158 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcb53e1-a58b-4c20-9ded-d2d7f6164a41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:43.728158 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcb53e1-a58b-4c20-9ded-d2d7f6164a41 admin admin] PATH_INFO: `/v3/roles` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:43.728841 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdcb53e1-a58b-4c20-9ded-d2d7f6164a41 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:43.730522 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdcb53e1-a58b-4c20-9ded-d2d7f6164a41 admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:43.738348 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1039/2062] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 16:00:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:00:43.740028 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5113dc5d-a7e3-4d55-93c7-c57137776dd6 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:43.741958 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1024/2063] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 30 16:00:43 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:00:43.745432 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcb53e1-a58b-4c20-9ded-d2d7f6164a41 admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:43.749729 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5113dc5d-a7e3-4d55-93c7-c57137776dd6 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:43.755983 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8c255d-c573-4536-bec2-b354aa2ff272 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:43.756874 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcb53e1-a58b-4c20-9ded-d2d7f6164a41 admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:43.757079 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8c255d-c573-4536-bec2-b354aa2ff272 admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:43.757405 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8c255d-c573-4536-bec2-b354aa2ff272 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:43.757713 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8c255d-c573-4536-bec2-b354aa2ff272 admin admin] PATH_INFO: `/v3/domains` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:43.758340 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8c255d-c573-4536-bec2-b354aa2ff272 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:00:43.758645 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8c255d-c573-4536-bec2-b354aa2ff272 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:43.760452 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8c255d-c573-4536-bec2-b354aa2ff272 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:43.762279 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298f9f61-0ae4-4974-8df3-94a90fa085eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:43.762760 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298f9f61-0ae4-4974-8df3-94a90fa085eb admin admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:43.763109 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298f9f61-0ae4-4974-8df3-94a90fa085eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:43.763461 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298f9f61-0ae4-4974-8df3-94a90fa085eb admin admin] PATH_INFO: `/v3/roles` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:43.764400 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298f9f61-0ae4-4974-8df3-94a90fa085eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:43.766072 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298f9f61-0ae4-4974-8df3-94a90fa085eb admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:43.777028 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1040/2064] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Aug 30 16:00:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:00:43.779343 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1025/2065] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 30 16:00:43 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:00:43.781787 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8c255d-c573-4536-bec2-b354aa2ff272 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:43.783638 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298f9f61-0ae4-4974-8df3-94a90fa085eb admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:43.796229 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8c255d-c573-4536-bec2-b354aa2ff272 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:43.802594 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173b1fbb-0e4b-42b6-87a6-48218c97f99a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:43.803539 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173b1fbb-0e4b-42b6-87a6-48218c97f99a admin admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:43.803797 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173b1fbb-0e4b-42b6-87a6-48218c97f99a admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:43.804141 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173b1fbb-0e4b-42b6-87a6-48218c97f99a admin admin] PATH_INFO: `/v3/projects` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:43.805507 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173b1fbb-0e4b-42b6-87a6-48218c97f99a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:43.808489 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173b1fbb-0e4b-42b6-87a6-48218c97f99a admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:43.813265 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298f9f61-0ae4-4974-8df3-94a90fa085eb admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:43.826352 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f242d0b2-d414-4154-90d5-5962004e969a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:43.827664 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f242d0b2-d414-4154-90d5-5962004e969a admin admin] REQUEST_METHOD: `PUT` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:43.828024 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f242d0b2-d414-4154-90d5-5962004e969a admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:43.828868 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f242d0b2-d414-4154-90d5-5962004e969a admin admin] PATH_INFO: `/v3/projects/cf03b164b1b443ada592eab5e9d2ef24/users/480baae5925343c1ac0bf0b87efcf9f7/roles/df8f9f844b1b439181d75a07c1a93418` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:43.862217 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1041/2066] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Aug 30 16:00:43 2022] POST /identity/v3/projects => generated 371 bytes in 80 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 16:00:43.863706 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173b1fbb-0e4b-42b6-87a6-48218c97f99a admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:43.870663 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f242d0b2-d414-4154-90d5-5962004e969a admin admin] RBAC: Authorizing `identity:create_grant(project_id=cf03b164b1b443ada592eab5e9d2ef24, user_id=480baae5925343c1ac0bf0b87efcf9f7, role_id=df8f9f844b1b439181d75a07c1a93418)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:43.875136 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f242d0b2-d414-4154-90d5-5962004e969a admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:43.901305 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1026/2067] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 30 16:00:43 2022] PUT /identity/v3/projects/cf03b164b1b443ada592eab5e9d2ef24/users/480baae5925343c1ac0bf0b87efcf9f7/roles/df8f9f844b1b439181d75a07c1a93418 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:00:43.909142 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555b2bbb-495c-40ae-904f-680f7d79b234 None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:43.909724 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555b2bbb-495c-40ae-904f-680f7d79b234 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:43.909994 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555b2bbb-495c-40ae-904f-680f7d79b234 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:43.946975 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-555b2bbb-495c-40ae-904f-680f7d79b234 None None] MFA Rules not processed for user `480baae5925343c1ac0bf0b87efcf9f7`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:00:43.948118 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-555b2bbb-495c-40ae-904f-680f7d79b234 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:00:43.972990 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173b1fbb-0e4b-42b6-87a6-48218c97f99a admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:44.031530 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1027/2068] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 16:00:43 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:00:44.036090 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555b2bbb-495c-40ae-904f-680f7d79b234 None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:44.059560 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67132c8c-666a-4bc4-ba34-0403bd381801 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:44.060682 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67132c8c-666a-4bc4-ba34-0403bd381801 admin admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:44.061107 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67132c8c-666a-4bc4-ba34-0403bd381801 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:44.061535 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67132c8c-666a-4bc4-ba34-0403bd381801 admin admin] PATH_INFO: `/v3/users` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:44.062635 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67132c8c-666a-4bc4-ba34-0403bd381801 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:44.065653 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67132c8c-666a-4bc4-ba34-0403bd381801 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:44.108106 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1042/2069] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Aug 30 16:00:43 2022] POST /identity/v3/users => generated 286 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:00:44.112269 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67132c8c-666a-4bc4-ba34-0403bd381801 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:44.123158 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67132c8c-666a-4bc4-ba34-0403bd381801 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:44.130284 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73aa9d17-29d6-46e1-8a5d-30e3bac09ac4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:44.130895 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73aa9d17-29d6-46e1-8a5d-30e3bac09ac4 admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:44.131169 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73aa9d17-29d6-46e1-8a5d-30e3bac09ac4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:44.131442 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73aa9d17-29d6-46e1-8a5d-30e3bac09ac4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:44.132092 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73aa9d17-29d6-46e1-8a5d-30e3bac09ac4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:44.133801 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73aa9d17-29d6-46e1-8a5d-30e3bac09ac4 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:44.147798 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1043/2070] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 30 16:00:44 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:00:44.148256 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73aa9d17-29d6-46e1-8a5d-30e3bac09ac4 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:44.149982 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-73aa9d17-29d6-46e1-8a5d-30e3bac09ac4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:00:44.152566 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555b2bbb-495c-40ae-904f-680f7d79b234 None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:44.163362 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73aa9d17-29d6-46e1-8a5d-30e3bac09ac4 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:44.163563 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452f8fa4-6db7-4087-bb95-8892d453bba0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:44.164878 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452f8fa4-6db7-4087-bb95-8892d453bba0 admin admin] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:44.165613 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452f8fa4-6db7-4087-bb95-8892d453bba0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:44.165974 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452f8fa4-6db7-4087-bb95-8892d453bba0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:44.166661 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452f8fa4-6db7-4087-bb95-8892d453bba0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:44.168811 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075c9610-fd6f-4d7b-ba6f-cd582df0264f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:44.168918 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452f8fa4-6db7-4087-bb95-8892d453bba0 admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:44.169451 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075c9610-fd6f-4d7b-ba6f-cd582df0264f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:44.169866 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075c9610-fd6f-4d7b-ba6f-cd582df0264f admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:44.170057 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075c9610-fd6f-4d7b-ba6f-cd582df0264f admin admin] PATH_INFO: `/v3/projects/2374ab8ff42d4e8296fd98dce3cf46bf` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:44.171250 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075c9610-fd6f-4d7b-ba6f-cd582df0264f admin admin] RBAC: Authorizing `identity:delete_project(project_id=2374ab8ff42d4e8296fd98dce3cf46bf)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:44.173381 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075c9610-fd6f-4d7b-ba6f-cd582df0264f admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:44.179691 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:00:44.201922 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1028/2071] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Aug 30 16:00:44 2022] POST /identity/v3/projects => generated 369 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:00:44.202755 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452f8fa4-6db7-4087-bb95-8892d453bba0 admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:44.216040 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:INSERT=5,keystone:SELECT=93,keystone:DELETE=2 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:00:44.232293 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-075c9610-fd6f-4d7b-ba6f-cd582df0264f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2374ab8ff42d4e8296fd98dce3cf46bf', 'request_id': 'req-075c9610-fd6f-4d7b-ba6f-cd582df0264f'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:44.247159 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-075c9610-fd6f-4d7b-ba6f-cd582df0264f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2374ab8ff42d4e8296fd98dce3cf46bf', 'request_id': 'req-075c9610-fd6f-4d7b-ba6f-cd582df0264f'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:44.256633 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-075c9610-fd6f-4d7b-ba6f-cd582df0264f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2374ab8ff42d4e8296fd98dce3cf46bf', 'request_id': 'req-075c9610-fd6f-4d7b-ba6f-cd582df0264f'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:44.267766 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-075c9610-fd6f-4d7b-ba6f-cd582df0264f admin admin] The token cache is being invalidate because project 2374ab8ff42d4e8296fd98dce3cf46bf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81511) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 16:00:44.271468 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-075c9610-fd6f-4d7b-ba6f-cd582df0264f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:44.271468 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-075c9610-fd6f-4d7b-ba6f-cd582df0264f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:44.271732 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452f8fa4-6db7-4087-bb95-8892d453bba0 admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:44.271836 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1044/2072] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Aug 30 16:00:44 2022] DELETE /identity/v3/projects/2374ab8ff42d4e8296fd98dce3cf46bf => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:00:44.271836 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075c9610-fd6f-4d7b-ba6f-cd582df0264f admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:44.274481 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-075c9610-fd6f-4d7b-ba6f-cd582df0264f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:00:44.333809 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa1c166-05da-47bf-8185-eccf69a85b1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:44.334427 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa1c166-05da-47bf-8185-eccf69a85b1f admin admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:44.334699 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa1c166-05da-47bf-8185-eccf69a85b1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:44.334969 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa1c166-05da-47bf-8185-eccf69a85b1f admin admin] PATH_INFO: `/v3/roles` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:44.335599 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa1c166-05da-47bf-8185-eccf69a85b1f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:44.337211 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa1c166-05da-47bf-8185-eccf69a85b1f admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:44.345751 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075c9610-fd6f-4d7b-ba6f-cd582df0264f admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:44.348535 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1029/2073] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 30 16:00:44 2022] GET /identity/v3/roles => generated 1420 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:00:44.349372 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa1c166-05da-47bf-8185-eccf69a85b1f admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:44.350387 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e904377-38d1-40e5-b0ef-af44bf059821 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:44.350953 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-eaa1c166-05da-47bf-8185-eccf69a85b1f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:00:44.351044 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e904377-38d1-40e5-b0ef-af44bf059821 admin admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:44.351247 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e904377-38d1-40e5-b0ef-af44bf059821 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:44.351781 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e904377-38d1-40e5-b0ef-af44bf059821 admin admin] PATH_INFO: `/v3/users` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:44.352289 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e904377-38d1-40e5-b0ef-af44bf059821 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:44.354273 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e904377-38d1-40e5-b0ef-af44bf059821 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:44.364296 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa1c166-05da-47bf-8185-eccf69a85b1f admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:44.369946 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07789244-25f3-474c-8607-3cc92c9ede0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:44.370662 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07789244-25f3-474c-8607-3cc92c9ede0f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:44.370942 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07789244-25f3-474c-8607-3cc92c9ede0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:44.371214 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07789244-25f3-474c-8607-3cc92c9ede0f admin admin] PATH_INFO: `/v3/users/28b6f8d640e5403f8d50cb4bb9d50448` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:44.372557 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07789244-25f3-474c-8607-3cc92c9ede0f admin admin] RBAC: Authorizing `identity:delete_user(user_id=28b6f8d640e5403f8d50cb4bb9d50448)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:44.374876 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07789244-25f3-474c-8607-3cc92c9ede0f admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:44.377791 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1045/2074] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Aug 30 16:00:44 2022] POST /identity/v3/users => generated 291 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:00:44.378657 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e904377-38d1-40e5-b0ef-af44bf059821 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:44.380770 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-8e904377-38d1-40e5-b0ef-af44bf059821 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:00:44.395316 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e904377-38d1-40e5-b0ef-af44bf059821 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:44.401083 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db888a4-b30c-47ad-b177-d7711eb89f29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:44.401766 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db888a4-b30c-47ad-b177-d7711eb89f29 admin admin] REQUEST_METHOD: `PUT` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:44.402041 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db888a4-b30c-47ad-b177-d7711eb89f29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:44.402311 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db888a4-b30c-47ad-b177-d7711eb89f29 admin admin] PATH_INFO: `/v3/projects/0cd9f7b3ede040a8830940f5c1651e5c/users/5ce5da4377f240b08df3f8e66e01c95b/roles/df8f9f844b1b439181d75a07c1a93418` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:44.417588 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db888a4-b30c-47ad-b177-d7711eb89f29 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0cd9f7b3ede040a8830940f5c1651e5c, user_id=5ce5da4377f240b08df3f8e66e01c95b, role_id=df8f9f844b1b439181d75a07c1a93418)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:44.420227 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db888a4-b30c-47ad-b177-d7711eb89f29 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:44.425595 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-07789244-25f3-474c-8607-3cc92c9ede0f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '28b6f8d640e5403f8d50cb4bb9d50448'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:44.433837 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-07789244-25f3-474c-8607-3cc92c9ede0f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '28b6f8d640e5403f8d50cb4bb9d50448'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:44.434965 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1046/2075] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 30 16:00:44 2022] PUT /identity/v3/projects/0cd9f7b3ede040a8830940f5c1651e5c/users/5ce5da4377f240b08df3f8e66e01c95b/roles/df8f9f844b1b439181d75a07c1a93418 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:00:44.435686 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db888a4-b30c-47ad-b177-d7711eb89f29 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:44.456817 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-07789244-25f3-474c-8607-3cc92c9ede0f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '28b6f8d640e5403f8d50cb4bb9d50448'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:44.463692 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-07789244-25f3-474c-8607-3cc92c9ede0f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '28b6f8d640e5403f8d50cb4bb9d50448'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:44.464615 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-07789244-25f3-474c-8607-3cc92c9ede0f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '28b6f8d640e5403f8d50cb4bb9d50448'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:44.470919 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1030/2076] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Aug 30 16:00:44 2022] DELETE /identity/v3/users/28b6f8d640e5403f8d50cb4bb9d50448 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:00:44.472130 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99cbe28-6813-45fb-b00c-9b7e76e05a6e None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:44.472405 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99cbe28-6813-45fb-b00c-9b7e76e05a6e None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:44.472672 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99cbe28-6813-45fb-b00c-9b7e76e05a6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:44.482556 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:00:44.499369 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db888a4-b30c-47ad-b177-d7711eb89f29 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:44.510097 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-a99cbe28-6813-45fb-b00c-9b7e76e05a6e None None] MFA Rules not processed for user `5ce5da4377f240b08df3f8e66e01c95b`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:00:44.511327 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-a99cbe28-6813-45fb-b00c-9b7e76e05a6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:00:44.532958 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=90,keystone:DELETE=5,keystone:INSERT=5 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:00:44.567998 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b0a14c-7f44-4e3b-9523-da0e0d2be968 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:44.568610 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b0a14c-7f44-4e3b-9523-da0e0d2be968 admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:44.568885 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b0a14c-7f44-4e3b-9523-da0e0d2be968 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:44.569151 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b0a14c-7f44-4e3b-9523-da0e0d2be968 admin admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:44.570324 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b0a14c-7f44-4e3b-9523-da0e0d2be968 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:44.571918 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b0a14c-7f44-4e3b-9523-da0e0d2be968 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:44.583261 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1047/2077] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 30 16:00:44 2022] GET /identity/v3/roles => generated 1420 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:00:44.588621 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b0a14c-7f44-4e3b-9523-da0e0d2be968 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:44.590575 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-27b0a14c-7f44-4e3b-9523-da0e0d2be968 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:00:44.601433 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1031/2078] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 16:00:44 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:00:44.602474 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99cbe28-6813-45fb-b00c-9b7e76e05a6e None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:44.604207 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-a99cbe28-6813-45fb-b00c-9b7e76e05a6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:00:44.605075 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b0a14c-7f44-4e3b-9523-da0e0d2be968 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:44.610365 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b14ea9-2d67-4f08-b4a3-41a4c2f2882c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:44.611100 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b14ea9-2d67-4f08-b4a3-41a4c2f2882c admin admin] REQUEST_METHOD: `PUT` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:44.611443 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b14ea9-2d67-4f08-b4a3-41a4c2f2882c admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:44.612142 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b14ea9-2d67-4f08-b4a3-41a4c2f2882c admin admin] PATH_INFO: `/v3/projects/7404694160dd43478d60906173c3a16f/users/701d11ac33764ad78aa321cfdc6f5528/roles/f265386950a641ac8413b53eda4ae34d` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:44.618700 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99cbe28-6813-45fb-b00c-9b7e76e05a6e None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:44.624292 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ae37b0-f2b0-4ac3-b238-46a6ed470dad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:44.624983 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ae37b0-f2b0-4ac3-b238-46a6ed470dad admin admin] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:44.625293 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ae37b0-f2b0-4ac3-b238-46a6ed470dad admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:44.625594 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ae37b0-f2b0-4ac3-b238-46a6ed470dad admin admin] PATH_INFO: `/v3/projects` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:44.631891 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ae37b0-f2b0-4ac3-b238-46a6ed470dad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:44.633947 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ae37b0-f2b0-4ac3-b238-46a6ed470dad admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:44.634620 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b14ea9-2d67-4f08-b4a3-41a4c2f2882c admin admin] RBAC: Authorizing `identity:create_grant(project_id=7404694160dd43478d60906173c3a16f, user_id=701d11ac33764ad78aa321cfdc6f5528, role_id=f265386950a641ac8413b53eda4ae34d)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:44.637400 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b14ea9-2d67-4f08-b4a3-41a4c2f2882c admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:44.661299 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1048/2079] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 30 16:00:44 2022] PUT /identity/v3/projects/7404694160dd43478d60906173c3a16f/users/701d11ac33764ad78aa321cfdc6f5528/roles/f265386950a641ac8413b53eda4ae34d => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:00:44.662450 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b14ea9-2d67-4f08-b4a3-41a4c2f2882c admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:44.675819 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1032/2080] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Aug 30 16:00:44 2022] POST /identity/v3/projects => generated 371 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:00:44.679993 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ae37b0-f2b0-4ac3-b238-46a6ed470dad admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:44.733063 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b14ea9-2d67-4f08-b4a3-41a4c2f2882c admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:44.745927 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ae37b0-f2b0-4ac3-b238-46a6ed470dad admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:44.751894 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca51ef5-2b95-4de8-94b0-18c4638e575a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:44.752780 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca51ef5-2b95-4de8-94b0-18c4638e575a admin admin] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:44.753104 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca51ef5-2b95-4de8-94b0-18c4638e575a admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:44.753413 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca51ef5-2b95-4de8-94b0-18c4638e575a admin admin] PATH_INFO: `/v3/users` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:44.754208 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca51ef5-2b95-4de8-94b0-18c4638e575a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:44.756347 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca51ef5-2b95-4de8-94b0-18c4638e575a admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:44.785885 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1033/2081] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Aug 30 16:00:44 2022] POST /identity/v3/users => generated 292 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:00:44.788178 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca51ef5-2b95-4de8-94b0-18c4638e575a admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:44.811324 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca51ef5-2b95-4de8-94b0-18c4638e575a admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:44.812192 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917c5a52-674c-487e-b09d-d547dba5b57d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:44.812981 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917c5a52-674c-487e-b09d-d547dba5b57d admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:44.813525 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917c5a52-674c-487e-b09d-d547dba5b57d admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:44.814325 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917c5a52-674c-487e-b09d-d547dba5b57d admin admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:44.815150 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917c5a52-674c-487e-b09d-d547dba5b57d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:44.817505 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917c5a52-674c-487e-b09d-d547dba5b57d admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:44.820237 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc7c08a-12a9-4677-a7d9-fa26b6f7660d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:44.821000 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc7c08a-12a9-4677-a7d9-fa26b6f7660d admin admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:44.821375 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc7c08a-12a9-4677-a7d9-fa26b6f7660d admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:44.821709 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc7c08a-12a9-4677-a7d9-fa26b6f7660d admin admin] PATH_INFO: `/v3/roles` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:44.822610 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc7c08a-12a9-4677-a7d9-fa26b6f7660d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:44.825134 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc7c08a-12a9-4677-a7d9-fa26b6f7660d admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:44.833937 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1049/2082] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 30 16:00:44 2022] GET /identity/v3/roles => generated 1420 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:00:44.841245 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1034/2083] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 30 16:00:44 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:00:44.842013 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc7c08a-12a9-4677-a7d9-fa26b6f7660d admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:44.842174 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917c5a52-674c-487e-b09d-d547dba5b57d admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:44.846067 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-917c5a52-674c-487e-b09d-d547dba5b57d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:00:44.853472 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc7c08a-12a9-4677-a7d9-fa26b6f7660d admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:44.858207 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf27759a-ffc8-4ced-a4b4-9e10a583fbd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:44.858862 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf27759a-ffc8-4ced-a4b4-9e10a583fbd1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:44.858987 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917c5a52-674c-487e-b09d-d547dba5b57d admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:44.859146 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf27759a-ffc8-4ced-a4b4-9e10a583fbd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:44.859421 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf27759a-ffc8-4ced-a4b4-9e10a583fbd1 admin admin] PATH_INFO: `/v3/projects/7404694160dd43478d60906173c3a16f/users/701d11ac33764ad78aa321cfdc6f5528/roles/f265386950a641ac8413b53eda4ae34d` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:44.863907 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c19ef1c-3393-47ff-b578-11cccfe4399f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:44.864279 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf27759a-ffc8-4ced-a4b4-9e10a583fbd1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7404694160dd43478d60906173c3a16f, user_id=701d11ac33764ad78aa321cfdc6f5528, role_id=f265386950a641ac8413b53eda4ae34d)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:44.864534 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c19ef1c-3393-47ff-b578-11cccfe4399f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:44.864810 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c19ef1c-3393-47ff-b578-11cccfe4399f admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:44.865080 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c19ef1c-3393-47ff-b578-11cccfe4399f admin admin] PATH_INFO: `/v3/projects/931345f1d3fb4d92b59750c6b979fbba` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:44.866232 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c19ef1c-3393-47ff-b578-11cccfe4399f admin admin] RBAC: Authorizing `identity:delete_project(project_id=931345f1d3fb4d92b59750c6b979fbba)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:44.867002 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf27759a-ffc8-4ced-a4b4-9e10a583fbd1 admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:44.868135 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c19ef1c-3393-47ff-b578-11cccfe4399f admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:44.879735 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1035/2084] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 30 16:00:44 2022] PUT /identity/v3/projects/7404694160dd43478d60906173c3a16f/users/701d11ac33764ad78aa321cfdc6f5528/roles/f265386950a641ac8413b53eda4ae34d => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:00:44.880522 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf27759a-ffc8-4ced-a4b4-9e10a583fbd1 admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:44.914499 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:00:44.919397 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-2c19ef1c-3393-47ff-b578-11cccfe4399f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '931345f1d3fb4d92b59750c6b979fbba', 'request_id': 'req-2c19ef1c-3393-47ff-b578-11cccfe4399f'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:44.921959 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:00:44.929685 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-2c19ef1c-3393-47ff-b578-11cccfe4399f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '931345f1d3fb4d92b59750c6b979fbba', 'request_id': 'req-2c19ef1c-3393-47ff-b578-11cccfe4399f'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:44.937974 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-2c19ef1c-3393-47ff-b578-11cccfe4399f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '931345f1d3fb4d92b59750c6b979fbba', 'request_id': 'req-2c19ef1c-3393-47ff-b578-11cccfe4399f'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:44.948774 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf27759a-ffc8-4ced-a4b4-9e10a583fbd1 admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:44.950807 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-2c19ef1c-3393-47ff-b578-11cccfe4399f admin admin] The token cache is being invalidate because project 931345f1d3fb4d92b59750c6b979fbba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81511) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 16:00:44.951105 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-2c19ef1c-3393-47ff-b578-11cccfe4399f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:44.951921 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-2c19ef1c-3393-47ff-b578-11cccfe4399f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:44.954006 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1050/2085] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Aug 30 16:00:44 2022] DELETE /identity/v3/projects/931345f1d3fb4d92b59750c6b979fbba => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:00:44.954980 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f652b31e-8a83-4121-9b5b-99b38848e394 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:44.955261 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f652b31e-8a83-4121-9b5b-99b38848e394 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:44.955600 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f652b31e-8a83-4121-9b5b-99b38848e394 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:44.987566 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-f652b31e-8a83-4121-9b5b-99b38848e394 None None] MFA Rules not processed for user `701d11ac33764ad78aa321cfdc6f5528`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:00:44.989117 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-f652b31e-8a83-4121-9b5b-99b38848e394 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:00:45.017893 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a45e562-4881-4578-b944-432ef96439aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:45.018559 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a45e562-4881-4578-b944-432ef96439aa admin admin] REQUEST_METHOD: `PUT` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:45.018839 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a45e562-4881-4578-b944-432ef96439aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:45.019111 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a45e562-4881-4578-b944-432ef96439aa admin admin] PATH_INFO: `/v3/projects/c0ddd4efe5344566a3b600b1c5e27120/users/667a371b0dd64c508cbf9681a8eb1dc9/roles/f265386950a641ac8413b53eda4ae34d` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:45.033389 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a45e562-4881-4578-b944-432ef96439aa admin admin] RBAC: Authorizing `identity:create_grant(project_id=c0ddd4efe5344566a3b600b1c5e27120, user_id=667a371b0dd64c508cbf9681a8eb1dc9, role_id=f265386950a641ac8413b53eda4ae34d)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:45.036065 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a45e562-4881-4578-b944-432ef96439aa admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:45.050556 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1036/2086] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 30 16:00:44 2022] PUT /identity/v3/projects/c0ddd4efe5344566a3b600b1c5e27120/users/667a371b0dd64c508cbf9681a8eb1dc9/roles/f265386950a641ac8413b53eda4ae34d => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:00:45.054363 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a45e562-4881-4578-b944-432ef96439aa admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:45.055990 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-9a45e562-4881-4578-b944-432ef96439aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:00:45.059299 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1051/2087] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 16:00:44 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:00:45.060314 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f652b31e-8a83-4121-9b5b-99b38848e394 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:45.063003 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-f652b31e-8a83-4121-9b5b-99b38848e394 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:00:45.078423 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f652b31e-8a83-4121-9b5b-99b38848e394 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:45.130017 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a45e562-4881-4578-b944-432ef96439aa admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:45.135577 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6682b310-3aaf-4527-b308-6973e5d5baf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:45.136217 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6682b310-3aaf-4527-b308-6973e5d5baf9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:45.136602 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6682b310-3aaf-4527-b308-6973e5d5baf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:45.136881 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6682b310-3aaf-4527-b308-6973e5d5baf9 admin admin] PATH_INFO: `/v3/users/548e1e36f955494cb7e9826aed29c292` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:45.138237 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6682b310-3aaf-4527-b308-6973e5d5baf9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=548e1e36f955494cb7e9826aed29c292)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:45.140427 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6682b310-3aaf-4527-b308-6973e5d5baf9 admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:45.143966 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e0108d-3a19-4f0f-a400-c4c801cab96e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:45.144660 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e0108d-3a19-4f0f-a400-c4c801cab96e admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:45.145003 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e0108d-3a19-4f0f-a400-c4c801cab96e admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:45.145366 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e0108d-3a19-4f0f-a400-c4c801cab96e admin admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:45.146082 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e0108d-3a19-4f0f-a400-c4c801cab96e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:45.147747 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e0108d-3a19-4f0f-a400-c4c801cab96e admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:45.161252 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1052/2088] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 30 16:00:45 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:00:45.165256 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e0108d-3a19-4f0f-a400-c4c801cab96e admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:45.167176 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-78e0108d-3a19-4f0f-a400-c4c801cab96e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:00:45.198206 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-6682b310-3aaf-4527-b308-6973e5d5baf9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '548e1e36f955494cb7e9826aed29c292'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:45.211698 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-6682b310-3aaf-4527-b308-6973e5d5baf9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '548e1e36f955494cb7e9826aed29c292'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:45.228318 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:00:45.230305 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-6682b310-3aaf-4527-b308-6973e5d5baf9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '548e1e36f955494cb7e9826aed29c292'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:45.237743 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e0108d-3a19-4f0f-a400-c4c801cab96e admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:45.242661 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1b8afc-d506-4121-a814-155346c930be service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0100a04821804993866ea20f159663f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:45.243132 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-6682b310-3aaf-4527-b308-6973e5d5baf9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '548e1e36f955494cb7e9826aed29c292'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:45.243553 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1b8afc-d506-4121-a814-155346c930be service cinder] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:45.243848 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-6682b310-3aaf-4527-b308-6973e5d5baf9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '548e1e36f955494cb7e9826aed29c292'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:45.244002 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1b8afc-d506-4121-a814-155346c930be service cinder] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:45.244366 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1b8afc-d506-4121-a814-155346c930be service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:45.246305 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-bd1b8afc-d506-4121-a814-155346c930be service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:00:45.251018 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1037/2089] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Aug 30 16:00:45 2022] DELETE /identity/v3/users/548e1e36f955494cb7e9826aed29c292 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:00:45.252521 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6682b310-3aaf-4527-b308-6973e5d5baf9 admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:45.254011 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:00:45.256099 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-6682b310-3aaf-4527-b308-6973e5d5baf9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:00:45.334745 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1b8afc-d506-4121-a814-155346c930be service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:45.336481 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1b8afc-d506-4121-a814-155346c930be service cinder] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:45.367235 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1053/2090] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 16:00:45 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:00:45.375149 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6682b310-3aaf-4527-b308-6973e5d5baf9 admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:45.382303 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622932de-3a1b-403d-ba7f-d1c57805b0d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:45.382980 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622932de-3a1b-403d-ba7f-d1c57805b0d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:45.383262 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622932de-3a1b-403d-ba7f-d1c57805b0d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:45.383951 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622932de-3a1b-403d-ba7f-d1c57805b0d2 admin admin] PATH_INFO: `/v3/projects/c0ddd4efe5344566a3b600b1c5e27120/users/667a371b0dd64c508cbf9681a8eb1dc9/roles/f265386950a641ac8413b53eda4ae34d` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:45.388443 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622932de-3a1b-403d-ba7f-d1c57805b0d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c0ddd4efe5344566a3b600b1c5e27120, user_id=667a371b0dd64c508cbf9681a8eb1dc9, role_id=f265386950a641ac8413b53eda4ae34d)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:45.391625 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622932de-3a1b-403d-ba7f-d1c57805b0d2 admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:45.407182 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1038/2091] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 30 16:00:45 2022] PUT /identity/v3/projects/c0ddd4efe5344566a3b600b1c5e27120/users/667a371b0dd64c508cbf9681a8eb1dc9/roles/f265386950a641ac8413b53eda4ae34d => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:00:45.419662 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c5c35d-12c1-4780-989d-07cadd7c2f27 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:45.419947 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c5c35d-12c1-4780-989d-07cadd7c2f27 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:45.420217 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c5c35d-12c1-4780-989d-07cadd7c2f27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:45.478255 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-98c5c35d-12c1-4780-989d-07cadd7c2f27 None None] MFA Rules not processed for user `667a371b0dd64c508cbf9681a8eb1dc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:00:45.481839 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-98c5c35d-12c1-4780-989d-07cadd7c2f27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:00:45.590578 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1054/2092] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 16:00:45 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:00:45.603681 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622932de-3a1b-403d-ba7f-d1c57805b0d2 admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:45.606538 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-622932de-3a1b-403d-ba7f-d1c57805b0d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:00:45.624862 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c5c35d-12c1-4780-989d-07cadd7c2f27 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:45.633550 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-98c5c35d-12c1-4780-989d-07cadd7c2f27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:00:45.729399 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622932de-3a1b-403d-ba7f-d1c57805b0d2 admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:45.735854 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f50d7f0-d4f6-4e24-b6ee-55875742eaef service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0100a04821804993866ea20f159663f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:45.736719 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f50d7f0-d4f6-4e24-b6ee-55875742eaef service cinder] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:45.738156 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f50d7f0-d4f6-4e24-b6ee-55875742eaef service cinder] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:45.738156 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f50d7f0-d4f6-4e24-b6ee-55875742eaef service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:45.739655 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c5c35d-12c1-4780-989d-07cadd7c2f27 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:45.749534 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483d36a4-5c82-45ee-928d-f405d263066b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:45.750599 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483d36a4-5c82-45ee-928d-f405d263066b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:45.751107 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483d36a4-5c82-45ee-928d-f405d263066b admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:45.751686 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483d36a4-5c82-45ee-928d-f405d263066b admin admin] PATH_INFO: `/v3/projects/fbeca0242d5b46e58a930d67c257c0d1` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:45.751882 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f50d7f0-d4f6-4e24-b6ee-55875742eaef service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:45.753718 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f50d7f0-d4f6-4e24-b6ee-55875742eaef service cinder] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:45.753998 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483d36a4-5c82-45ee-928d-f405d263066b admin admin] RBAC: Authorizing `identity:delete_project(project_id=fbeca0242d5b46e58a930d67c257c0d1)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:45.756751 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483d36a4-5c82-45ee-928d-f405d263066b admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:45.768527 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1039/2093] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 16:00:45 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:00:45.811489 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:00:45.830775 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-483d36a4-5c82-45ee-928d-f405d263066b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fbeca0242d5b46e58a930d67c257c0d1', 'request_id': 'req-483d36a4-5c82-45ee-928d-f405d263066b'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:45.846662 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-483d36a4-5c82-45ee-928d-f405d263066b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fbeca0242d5b46e58a930d67c257c0d1', 'request_id': 'req-483d36a4-5c82-45ee-928d-f405d263066b'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:45.856152 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-483d36a4-5c82-45ee-928d-f405d263066b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fbeca0242d5b46e58a930d67c257c0d1', 'request_id': 'req-483d36a4-5c82-45ee-928d-f405d263066b'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:45.868463 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-483d36a4-5c82-45ee-928d-f405d263066b admin admin] The token cache is being invalidate because project fbeca0242d5b46e58a930d67c257c0d1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81511) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 16:00:45.868779 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-483d36a4-5c82-45ee-928d-f405d263066b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:45.869651 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-483d36a4-5c82-45ee-928d-f405d263066b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:45.871765 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1055/2094] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Aug 30 16:00:45 2022] DELETE /identity/v3/projects/fbeca0242d5b46e58a930d67c257c0d1 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:00:45.878657 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41651414-12f7-42e8-8c0b-ae92eaaedcd4 None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:45.878850 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41651414-12f7-42e8-8c0b-ae92eaaedcd4 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:45.879128 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41651414-12f7-42e8-8c0b-ae92eaaedcd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:45.901678 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-41651414-12f7-42e8-8c0b-ae92eaaedcd4 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:00:45.902449 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-41651414-12f7-42e8-8c0b-ae92eaaedcd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:00:45.995338 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1040/2095] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 16:00:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:00:46.005151 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c57452-3fe0-4525-96ec-26df58ee894f None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:46.005465 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c57452-3fe0-4525-96ec-26df58ee894f None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:46.005740 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c57452-3fe0-4525-96ec-26df58ee894f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:46.031007 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-a9c57452-3fe0-4525-96ec-26df58ee894f None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:00:46.032757 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-a9c57452-3fe0-4525-96ec-26df58ee894f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:00:46.041958 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1056/2096] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 16:00:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:00:46.057699 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41651414-12f7-42e8-8c0b-ae92eaaedcd4 None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:46.064200 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41651414-12f7-42e8-8c0b-ae92eaaedcd4 None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:46.069699 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170988e7-347f-415f-be46-b72082db5f56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:46.070383 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170988e7-347f-415f-be46-b72082db5f56 admin admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:46.070746 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170988e7-347f-415f-be46-b72082db5f56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:46.071055 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170988e7-347f-415f-be46-b72082db5f56 admin admin] PATH_INFO: `/v3/domains` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:46.071826 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170988e7-347f-415f-be46-b72082db5f56 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81512) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:00:46.072134 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170988e7-347f-415f-be46-b72082db5f56 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:46.074677 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170988e7-347f-415f-be46-b72082db5f56 admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:46.089785 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1041/2097] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Aug 30 16:00:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:00:46.097144 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c57452-3fe0-4525-96ec-26df58ee894f None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:46.107428 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c57452-3fe0-4525-96ec-26df58ee894f None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:46.113395 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a387c603-92ac-4d82-9fff-865f20920bfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:46.114184 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a387c603-92ac-4d82-9fff-865f20920bfe admin admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:46.114339 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a387c603-92ac-4d82-9fff-865f20920bfe admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:46.114614 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a387c603-92ac-4d82-9fff-865f20920bfe admin admin] PATH_INFO: `/v3/projects` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:46.115255 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a387c603-92ac-4d82-9fff-865f20920bfe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:46.117485 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a387c603-92ac-4d82-9fff-865f20920bfe admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:46.158172 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1057/2098] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Aug 30 16:00:46 2022] POST /identity/v3/projects => generated 367 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:00:46.164124 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170988e7-347f-415f-be46-b72082db5f56 admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:46.177149 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170988e7-347f-415f-be46-b72082db5f56 admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:46.196780 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:00:46.252606 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75dd3ddf-e595-471d-a470-a5931bc67f37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:46.253405 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dd3ddf-e595-471d-a470-a5931bc67f37 admin admin] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:46.259545 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dd3ddf-e595-471d-a470-a5931bc67f37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:46.259545 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dd3ddf-e595-471d-a470-a5931bc67f37 admin admin] PATH_INFO: `/v3/users` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:46.259545 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75dd3ddf-e595-471d-a470-a5931bc67f37 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:46.259545 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75dd3ddf-e595-471d-a470-a5931bc67f37 admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:46.289737 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1042/2099] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Aug 30 16:00:46 2022] POST /identity/v3/users => generated 284 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:00:46.293809 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a387c603-92ac-4d82-9fff-865f20920bfe admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:46.304617 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a387c603-92ac-4d82-9fff-865f20920bfe admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:46.309282 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f30d3d9-8995-47f3-b220-0fdc1f0d4e72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:46.309901 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f30d3d9-8995-47f3-b220-0fdc1f0d4e72 admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:46.310224 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f30d3d9-8995-47f3-b220-0fdc1f0d4e72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:46.310529 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f30d3d9-8995-47f3-b220-0fdc1f0d4e72 admin admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:46.311186 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f30d3d9-8995-47f3-b220-0fdc1f0d4e72 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:46.312800 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f30d3d9-8995-47f3-b220-0fdc1f0d4e72 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:46.333162 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1058/2100] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 30 16:00:46 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:00:46.337809 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75dd3ddf-e595-471d-a470-a5931bc67f37 admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:46.352881 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75dd3ddf-e595-471d-a470-a5931bc67f37 admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:46.361332 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714d38bb-f9c6-4e53-a41e-3362acfa2d8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:46.361942 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714d38bb-f9c6-4e53-a41e-3362acfa2d8f admin admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:46.362349 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714d38bb-f9c6-4e53-a41e-3362acfa2d8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:46.362616 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714d38bb-f9c6-4e53-a41e-3362acfa2d8f admin admin] PATH_INFO: `/v3/roles` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:46.363610 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714d38bb-f9c6-4e53-a41e-3362acfa2d8f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:46.365722 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714d38bb-f9c6-4e53-a41e-3362acfa2d8f admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:46.381268 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1043/2101] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 30 16:00:46 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 16:00:46.385502 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f30d3d9-8995-47f3-b220-0fdc1f0d4e72 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:46.401409 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714d38bb-f9c6-4e53-a41e-3362acfa2d8f admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:46.401681 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f30d3d9-8995-47f3-b220-0fdc1f0d4e72 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:46.403384 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-714d38bb-f9c6-4e53-a41e-3362acfa2d8f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:00:46.406613 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eff2087-5584-4056-bae9-e5bd52af9c74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:46.407318 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eff2087-5584-4056-bae9-e5bd52af9c74 admin admin] REQUEST_METHOD: `PUT` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:46.408089 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eff2087-5584-4056-bae9-e5bd52af9c74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:46.408089 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eff2087-5584-4056-bae9-e5bd52af9c74 admin admin] PATH_INFO: `/v3/projects/3d00d1a960d04391b44d0ae5084df605/users/b2754da700034d59a225c68f365693ce/roles/df8f9f844b1b439181d75a07c1a93418` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:46.417852 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714d38bb-f9c6-4e53-a41e-3362acfa2d8f admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:46.425617 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a763e440-38b8-49d8-982e-ef9e3092a187 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:46.426590 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eff2087-5584-4056-bae9-e5bd52af9c74 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3d00d1a960d04391b44d0ae5084df605, user_id=b2754da700034d59a225c68f365693ce, role_id=df8f9f844b1b439181d75a07c1a93418)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:46.426745 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a763e440-38b8-49d8-982e-ef9e3092a187 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:46.427204 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a763e440-38b8-49d8-982e-ef9e3092a187 admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:46.427642 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a763e440-38b8-49d8-982e-ef9e3092a187 admin admin] PATH_INFO: `/v3/users/5ce5da4377f240b08df3f8e66e01c95b` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:46.428819 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a763e440-38b8-49d8-982e-ef9e3092a187 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5ce5da4377f240b08df3f8e66e01c95b)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:46.430900 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eff2087-5584-4056-bae9-e5bd52af9c74 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:46.431087 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a763e440-38b8-49d8-982e-ef9e3092a187 admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:46.460086 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1059/2102] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 30 16:00:46 2022] PUT /identity/v3/projects/3d00d1a960d04391b44d0ae5084df605/users/b2754da700034d59a225c68f365693ce/roles/df8f9f844b1b439181d75a07c1a93418 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:00:46.472893 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2ea3a2-7694-47b5-ab1a-30f6b9a9bb9d None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:46.473232 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2ea3a2-7694-47b5-ab1a-30f6b9a9bb9d None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:46.473482 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2ea3a2-7694-47b5-ab1a-30f6b9a9bb9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:46.505556 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-a763e440-38b8-49d8-982e-ef9e3092a187 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5ce5da4377f240b08df3f8e66e01c95b'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:46.519344 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-a763e440-38b8-49d8-982e-ef9e3092a187 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5ce5da4377f240b08df3f8e66e01c95b'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:46.520500 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-7f2ea3a2-7694-47b5-ab1a-30f6b9a9bb9d None None] MFA Rules not processed for user `b2754da700034d59a225c68f365693ce`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:00:46.522669 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-7f2ea3a2-7694-47b5-ab1a-30f6b9a9bb9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:00:46.540056 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-a763e440-38b8-49d8-982e-ef9e3092a187 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5ce5da4377f240b08df3f8e66e01c95b'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:46.551016 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-a763e440-38b8-49d8-982e-ef9e3092a187 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5ce5da4377f240b08df3f8e66e01c95b'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:46.552904 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-a763e440-38b8-49d8-982e-ef9e3092a187 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5ce5da4377f240b08df3f8e66e01c95b'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:46.566358 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1044/2103] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Aug 30 16:00:46 2022] DELETE /identity/v3/users/5ce5da4377f240b08df3f8e66e01c95b => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 16:00:46.571885 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf5b99b-29cd-4508-8cc5-ebc128eb26a6 None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:46.573640 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf5b99b-29cd-4508-8cc5-ebc128eb26a6 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:46.574293 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf5b99b-29cd-4508-8cc5-ebc128eb26a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:46.598507 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-7cf5b99b-29cd-4508-8cc5-ebc128eb26a6 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:00:46.599660 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-7cf5b99b-29cd-4508-8cc5-ebc128eb26a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:00:46.629590 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1060/2104] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 16:00:46 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 16:00:46.635964 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2ea3a2-7694-47b5-ab1a-30f6b9a9bb9d None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:46.637655 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-7f2ea3a2-7694-47b5-ab1a-30f6b9a9bb9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:00:46.675272 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1045/2105] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 16:00:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 16:00:46.687009 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf5b99b-29cd-4508-8cc5-ebc128eb26a6 None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:46.688608 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-7cf5b99b-29cd-4508-8cc5-ebc128eb26a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:00:46.695255 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:00:46.699355 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:00:46.721071 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2ea3a2-7694-47b5-ab1a-30f6b9a9bb9d None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:46.727474 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa51109-2a89-4149-af96-1bbd713ff42b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:46.728259 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa51109-2a89-4149-af96-1bbd713ff42b admin admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:46.728500 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa51109-2a89-4149-af96-1bbd713ff42b admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:46.728769 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa51109-2a89-4149-af96-1bbd713ff42b admin admin] PATH_INFO: `/v3/projects` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:46.729476 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa51109-2a89-4149-af96-1bbd713ff42b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:46.731808 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa51109-2a89-4149-af96-1bbd713ff42b admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:46.761759 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf5b99b-29cd-4508-8cc5-ebc128eb26a6 None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:46.766332 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adff95bc-0e4a-4d1e-9bb4-e83d3377c02d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f4157105f3488f807b7475008ab722', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:46.767070 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adff95bc-0e4a-4d1e-9bb4-e83d3377c02d service neutron] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:46.767349 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adff95bc-0e4a-4d1e-9bb4-e83d3377c02d service neutron] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:46.767615 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adff95bc-0e4a-4d1e-9bb4-e83d3377c02d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:46.771089 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1061/2106] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Aug 30 16:00:46 2022] POST /identity/v3/projects => generated 365 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:00:46.772027 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa51109-2a89-4149-af96-1bbd713ff42b admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:46.779563 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adff95bc-0e4a-4d1e-9bb4-e83d3377c02d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:46.781038 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adff95bc-0e4a-4d1e-9bb4-e83d3377c02d service neutron] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:46.791351 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1046/2107] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 16:00:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 16:00:46.853302 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa51109-2a89-4149-af96-1bbd713ff42b admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:46.861181 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff355ff-11ec-4fcd-b5f7-057e91375a07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:46.861835 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff355ff-11ec-4fcd-b5f7-057e91375a07 admin admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:46.862115 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff355ff-11ec-4fcd-b5f7-057e91375a07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:46.862387 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff355ff-11ec-4fcd-b5f7-057e91375a07 admin admin] PATH_INFO: `/v3/users` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:46.863363 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff355ff-11ec-4fcd-b5f7-057e91375a07 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:46.865511 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff355ff-11ec-4fcd-b5f7-057e91375a07 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:46.900212 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1062/2108] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Aug 30 16:00:46 2022] POST /identity/v3/users => generated 289 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 16:00:46.905761 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adff95bc-0e4a-4d1e-9bb4-e83d3377c02d service neutron] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:46.918369 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adff95bc-0e4a-4d1e-9bb4-e83d3377c02d service neutron] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:46.924820 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee4e477-3874-42e6-9a94-753a9f61f63c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:46.925514 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee4e477-3874-42e6-9a94-753a9f61f63c admin admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:46.925765 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee4e477-3874-42e6-9a94-753a9f61f63c admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:46.926032 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee4e477-3874-42e6-9a94-753a9f61f63c admin admin] PATH_INFO: `/v3/roles` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:46.926884 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee4e477-3874-42e6-9a94-753a9f61f63c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:46.928525 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee4e477-3874-42e6-9a94-753a9f61f63c admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:46.939819 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1047/2109] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 30 16:00:46 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:00:46.943843 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff355ff-11ec-4fcd-b5f7-057e91375a07 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:46.954352 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff355ff-11ec-4fcd-b5f7-057e91375a07 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:46.962030 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bda6054-471e-44da-a181-5a5e03b34f8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:46.962813 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bda6054-471e-44da-a181-5a5e03b34f8f admin admin] REQUEST_METHOD: `PUT` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:46.963387 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bda6054-471e-44da-a181-5a5e03b34f8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:46.968131 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bda6054-471e-44da-a181-5a5e03b34f8f admin admin] PATH_INFO: `/v3/projects/de0cb52f3308495f919adf22f646983f/users/82bcfe79999e4cd4a1d27c60c27798a8/roles/f265386950a641ac8413b53eda4ae34d` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:46.989934 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bda6054-471e-44da-a181-5a5e03b34f8f admin admin] RBAC: Authorizing `identity:create_grant(project_id=de0cb52f3308495f919adf22f646983f, user_id=82bcfe79999e4cd4a1d27c60c27798a8, role_id=f265386950a641ac8413b53eda4ae34d)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:46.992759 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bda6054-471e-44da-a181-5a5e03b34f8f admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:47.012278 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1063/2110] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 30 16:00:46 2022] PUT /identity/v3/projects/de0cb52f3308495f919adf22f646983f/users/82bcfe79999e4cd4a1d27c60c27798a8/roles/f265386950a641ac8413b53eda4ae34d => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:00:47.016220 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee4e477-3874-42e6-9a94-753a9f61f63c admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:47.030555 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee4e477-3874-42e6-9a94-753a9f61f63c admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:47.121063 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1a0dce-43f5-450b-80a3-3f506fd4555f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:47.121705 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1a0dce-43f5-450b-80a3-3f506fd4555f admin admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:47.121983 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1a0dce-43f5-450b-80a3-3f506fd4555f admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:47.122299 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1a0dce-43f5-450b-80a3-3f506fd4555f admin admin] PATH_INFO: `/v3/roles` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:47.122944 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1a0dce-43f5-450b-80a3-3f506fd4555f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:47.124643 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1a0dce-43f5-450b-80a3-3f506fd4555f admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:47.135691 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1048/2111] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 30 16:00:47 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:00:47.140528 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bda6054-471e-44da-a181-5a5e03b34f8f admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:47.153471 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bda6054-471e-44da-a181-5a5e03b34f8f admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:47.159575 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea3960a-59bb-41df-8119-30985c2df09b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:47.160308 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea3960a-59bb-41df-8119-30985c2df09b admin admin] REQUEST_METHOD: `PUT` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:47.160872 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea3960a-59bb-41df-8119-30985c2df09b admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:47.161050 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea3960a-59bb-41df-8119-30985c2df09b admin admin] PATH_INFO: `/v3/projects/de0cb52f3308495f919adf22f646983f/users/82bcfe79999e4cd4a1d27c60c27798a8/roles/f265386950a641ac8413b53eda4ae34d` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:47.163639 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea3960a-59bb-41df-8119-30985c2df09b admin admin] RBAC: Authorizing `identity:create_grant(project_id=de0cb52f3308495f919adf22f646983f, user_id=82bcfe79999e4cd4a1d27c60c27798a8, role_id=f265386950a641ac8413b53eda4ae34d)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:47.166642 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea3960a-59bb-41df-8119-30985c2df09b admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:47.181276 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1064/2112] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 30 16:00:47 2022] PUT /identity/v3/projects/de0cb52f3308495f919adf22f646983f/users/82bcfe79999e4cd4a1d27c60c27798a8/roles/f265386950a641ac8413b53eda4ae34d => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:00:47.189945 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93472dee-1bb8-47d0-94f8-857399ef8884 None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:47.190256 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93472dee-1bb8-47d0-94f8-857399ef8884 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:47.190507 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93472dee-1bb8-47d0-94f8-857399ef8884 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:47.213646 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea3960a-59bb-41df-8119-30985c2df09b admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:47.216241 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-3ea3960a-59bb-41df-8119-30985c2df09b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:00:47.228239 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-93472dee-1bb8-47d0-94f8-857399ef8884 None None] MFA Rules not processed for user `82bcfe79999e4cd4a1d27c60c27798a8`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:00:47.230665 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-93472dee-1bb8-47d0-94f8-857399ef8884 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:00:47.297947 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea3960a-59bb-41df-8119-30985c2df09b admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:47.303973 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8449a4-f4b4-4850-9441-9795095dfe39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:47.304692 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8449a4-f4b4-4850-9441-9795095dfe39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:47.304982 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8449a4-f4b4-4850-9441-9795095dfe39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:47.305308 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8449a4-f4b4-4850-9441-9795095dfe39 admin admin] PATH_INFO: `/v3/projects/0cd9f7b3ede040a8830940f5c1651e5c` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:47.306634 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8449a4-f4b4-4850-9441-9795095dfe39 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0cd9f7b3ede040a8830940f5c1651e5c)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:47.308746 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8449a4-f4b4-4850-9441-9795095dfe39 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:47.337402 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1049/2113] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 16:00:47 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:00:47.342912 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93472dee-1bb8-47d0-94f8-857399ef8884 None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:47.344492 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-93472dee-1bb8-47d0-94f8-857399ef8884 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:00:47.345729 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=93,keystone:INSERT=6,keystone:DELETE=1 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:00:47.361384 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93472dee-1bb8-47d0-94f8-857399ef8884 None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:47.371377 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-ae8449a4-f4b4-4850-9441-9795095dfe39 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0cd9f7b3ede040a8830940f5c1651e5c', 'request_id': 'req-ae8449a4-f4b4-4850-9441-9795095dfe39'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:47.374145 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:00:47.379773 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-ae8449a4-f4b4-4850-9441-9795095dfe39 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0cd9f7b3ede040a8830940f5c1651e5c', 'request_id': 'req-ae8449a4-f4b4-4850-9441-9795095dfe39'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:47.386499 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-ae8449a4-f4b4-4850-9441-9795095dfe39 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0cd9f7b3ede040a8830940f5c1651e5c', 'request_id': 'req-ae8449a4-f4b4-4850-9441-9795095dfe39'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:47.397360 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-ae8449a4-f4b4-4850-9441-9795095dfe39 admin admin] The token cache is being invalidate because project 0cd9f7b3ede040a8830940f5c1651e5c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81511) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 16:00:47.397650 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-ae8449a4-f4b4-4850-9441-9795095dfe39 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:47.398329 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-ae8449a4-f4b4-4850-9441-9795095dfe39 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:47.399851 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1065/2114] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Aug 30 16:00:47 2022] DELETE /identity/v3/projects/0cd9f7b3ede040a8830940f5c1651e5c => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 16:00:47.403338 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8449a4-f4b4-4850-9441-9795095dfe39 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:47.403338 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-ae8449a4-f4b4-4850-9441-9795095dfe39 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:00:47.419999 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c576c9-87aa-4512-baf4-de8aa16df04b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:47.420636 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c576c9-87aa-4512-baf4-de8aa16df04b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:47.420916 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c576c9-87aa-4512-baf4-de8aa16df04b admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:47.421191 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c576c9-87aa-4512-baf4-de8aa16df04b admin admin] PATH_INFO: `/v3/users/480baae5925343c1ac0bf0b87efcf9f7` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:47.422451 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c576c9-87aa-4512-baf4-de8aa16df04b admin admin] RBAC: Authorizing `identity:delete_user(user_id=480baae5925343c1ac0bf0b87efcf9f7)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:47.424415 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c576c9-87aa-4512-baf4-de8aa16df04b admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:47.466797 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8449a4-f4b4-4850-9441-9795095dfe39 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:47.470645 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6d33cc-8ef1-4a1d-89e3-b8346082b9a0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0100a04821804993866ea20f159663f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:47.471345 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6d33cc-8ef1-4a1d-89e3-b8346082b9a0 service cinder] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:47.471737 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6d33cc-8ef1-4a1d-89e3-b8346082b9a0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:47.472098 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6d33cc-8ef1-4a1d-89e3-b8346082b9a0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:47.474202 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-eb6d33cc-8ef1-4a1d-89e3-b8346082b9a0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:00:47.477786 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-f3c576c9-87aa-4512-baf4-de8aa16df04b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '480baae5925343c1ac0bf0b87efcf9f7'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:47.486322 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-f3c576c9-87aa-4512-baf4-de8aa16df04b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '480baae5925343c1ac0bf0b87efcf9f7'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:47.500126 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-f3c576c9-87aa-4512-baf4-de8aa16df04b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '480baae5925343c1ac0bf0b87efcf9f7'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:47.508398 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-f3c576c9-87aa-4512-baf4-de8aa16df04b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '480baae5925343c1ac0bf0b87efcf9f7'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:47.509146 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-f3c576c9-87aa-4512-baf4-de8aa16df04b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '480baae5925343c1ac0bf0b87efcf9f7'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:47.515413 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1050/2115] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Aug 30 16:00:47 2022] DELETE /identity/v3/users/480baae5925343c1ac0bf0b87efcf9f7 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 16:00:47.516402 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c576c9-87aa-4512-baf4-de8aa16df04b admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:47.517880 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-f3c576c9-87aa-4512-baf4-de8aa16df04b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:00:47.539833 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6d33cc-8ef1-4a1d-89e3-b8346082b9a0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:47.541347 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6d33cc-8ef1-4a1d-89e3-b8346082b9a0 service cinder] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:47.551309 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1066/2116] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 16:00:47 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:00:47.552625 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a61335-dc93-4bdb-bc22-3d3f63346638 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:47.552909 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a61335-dc93-4bdb-bc22-3d3f63346638 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:47.553259 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a61335-dc93-4bdb-bc22-3d3f63346638 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:47.572903 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-d0a61335-dc93-4bdb-bc22-3d3f63346638 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:00:47.574465 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-d0a61335-dc93-4bdb-bc22-3d3f63346638 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:00:47.583123 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c576c9-87aa-4512-baf4-de8aa16df04b admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:47.587316 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1067/2117] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 16:00:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:00:47.595361 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a61335-dc93-4bdb-bc22-3d3f63346638 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:47.597442 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-d0a61335-dc93-4bdb-bc22-3d3f63346638 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:00:47.611568 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31eb3b81-9d70-4fc4-8efe-64f51eab4fd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:47.612233 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31eb3b81-9d70-4fc4-8efe-64f51eab4fd4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:47.612516 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31eb3b81-9d70-4fc4-8efe-64f51eab4fd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:47.613145 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31eb3b81-9d70-4fc4-8efe-64f51eab4fd4 admin admin] PATH_INFO: `/v3/users/667a371b0dd64c508cbf9681a8eb1dc9` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:47.618273 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31eb3b81-9d70-4fc4-8efe-64f51eab4fd4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=667a371b0dd64c508cbf9681a8eb1dc9)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:47.621011 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31eb3b81-9d70-4fc4-8efe-64f51eab4fd4 admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:47.686476 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a61335-dc93-4bdb-bc22-3d3f63346638 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:47.690652 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6885f5e6-6101-47b5-9fc8-00662a03fa9b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f4157105f3488f807b7475008ab722', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:47.691280 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6885f5e6-6101-47b5-9fc8-00662a03fa9b service neutron] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:47.691737 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6885f5e6-6101-47b5-9fc8-00662a03fa9b service neutron] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:47.692009 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6885f5e6-6101-47b5-9fc8-00662a03fa9b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:47.700161 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:00:47.703767 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6885f5e6-6101-47b5-9fc8-00662a03fa9b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:47.705488 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6885f5e6-6101-47b5-9fc8-00662a03fa9b service neutron] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:47.709344 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-31eb3b81-9d70-4fc4-8efe-64f51eab4fd4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '667a371b0dd64c508cbf9681a8eb1dc9'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:47.713277 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:00:47.720945 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1068/2118] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 16:00:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:00:47.721711 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6885f5e6-6101-47b5-9fc8-00662a03fa9b service neutron] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:47.724221 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-31eb3b81-9d70-4fc4-8efe-64f51eab4fd4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '667a371b0dd64c508cbf9681a8eb1dc9'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:47.726044 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-6885f5e6-6101-47b5-9fc8-00662a03fa9b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:00:47.741080 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6885f5e6-6101-47b5-9fc8-00662a03fa9b service neutron] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:47.748088 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbb126a-cdcb-41aa-bc79-497864a25fa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:47.748830 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbb126a-cdcb-41aa-bc79-497864a25fa2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:47.749177 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbb126a-cdcb-41aa-bc79-497864a25fa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:47.749538 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-31eb3b81-9d70-4fc4-8efe-64f51eab4fd4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '667a371b0dd64c508cbf9681a8eb1dc9'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:47.750431 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbb126a-cdcb-41aa-bc79-497864a25fa2 admin admin] PATH_INFO: `/v3/users/1a8a5ef37e0e4bb48b40e542aa376b3a` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:47.751918 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbbb126a-cdcb-41aa-bc79-497864a25fa2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1a8a5ef37e0e4bb48b40e542aa376b3a)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:47.754242 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbbb126a-cdcb-41aa-bc79-497864a25fa2 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:47.760541 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-31eb3b81-9d70-4fc4-8efe-64f51eab4fd4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '667a371b0dd64c508cbf9681a8eb1dc9'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:47.763024 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-31eb3b81-9d70-4fc4-8efe-64f51eab4fd4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '667a371b0dd64c508cbf9681a8eb1dc9'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:47.776428 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1051/2119] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Aug 30 16:00:47 2022] DELETE /identity/v3/users/667a371b0dd64c508cbf9681a8eb1dc9 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:00:47.776428 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31eb3b81-9d70-4fc4-8efe-64f51eab4fd4 admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:47.776428 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-31eb3b81-9d70-4fc4-8efe-64f51eab4fd4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:00:47.839275 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-dbbb126a-cdcb-41aa-bc79-497864a25fa2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1a8a5ef37e0e4bb48b40e542aa376b3a'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:47.867932 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31eb3b81-9d70-4fc4-8efe-64f51eab4fd4 admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:47.868713 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-dbbb126a-cdcb-41aa-bc79-497864a25fa2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1a8a5ef37e0e4bb48b40e542aa376b3a'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:47.872245 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8f2c1d-0bbd-4c01-818f-e9f0da446e46 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0100a04821804993866ea20f159663f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:47.872870 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8f2c1d-0bbd-4c01-818f-e9f0da446e46 service cinder] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:47.873256 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8f2c1d-0bbd-4c01-818f-e9f0da446e46 service cinder] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:47.873716 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8f2c1d-0bbd-4c01-818f-e9f0da446e46 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:47.875995 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-2e8f2c1d-0bbd-4c01-818f-e9f0da446e46 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:00:47.885955 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-dbbb126a-cdcb-41aa-bc79-497864a25fa2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1a8a5ef37e0e4bb48b40e542aa376b3a'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:47.887029 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-dbbb126a-cdcb-41aa-bc79-497864a25fa2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1a8a5ef37e0e4bb48b40e542aa376b3a'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:47.901459 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-dbbb126a-cdcb-41aa-bc79-497864a25fa2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1a8a5ef37e0e4bb48b40e542aa376b3a'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:47.913346 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1069/2120] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Aug 30 16:00:47 2022] DELETE /identity/v3/users/1a8a5ef37e0e4bb48b40e542aa376b3a => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:00:47.917080 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2787a01a-4017-4c47-a476-7ba4c52fd172 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:47.917669 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2787a01a-4017-4c47-a476-7ba4c52fd172 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:47.919382 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2787a01a-4017-4c47-a476-7ba4c52fd172 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:47.954386 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-2787a01a-4017-4c47-a476-7ba4c52fd172 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:00:47.957976 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-2787a01a-4017-4c47-a476-7ba4c52fd172 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:00:47.984020 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8f2c1d-0bbd-4c01-818f-e9f0da446e46 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:47.988135 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8f2c1d-0bbd-4c01-818f-e9f0da446e46 service cinder] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:48.007651 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1052/2121] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 16:00:47 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 234 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:00:48.056375 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1070/2122] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 16:00:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 16:00:48.085097 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8f2c1d-0bbd-4c01-818f-e9f0da446e46 service cinder] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:48.086658 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-2e8f2c1d-0bbd-4c01-818f-e9f0da446e46 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:00:48.152979 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2787a01a-4017-4c47-a476-7ba4c52fd172 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:48.159152 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-2787a01a-4017-4c47-a476-7ba4c52fd172 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:00:48.191507 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2787a01a-4017-4c47-a476-7ba4c52fd172 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:48.204056 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d929ba55-8f3e-4dee-befe-96cf93ba1372 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:48.204834 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d929ba55-8f3e-4dee-befe-96cf93ba1372 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:48.205408 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d929ba55-8f3e-4dee-befe-96cf93ba1372 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:48.205726 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d929ba55-8f3e-4dee-befe-96cf93ba1372 admin admin] PATH_INFO: `/v3/projects/c0ddd4efe5344566a3b600b1c5e27120` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:48.208301 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d929ba55-8f3e-4dee-befe-96cf93ba1372 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c0ddd4efe5344566a3b600b1c5e27120)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:48.210811 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d929ba55-8f3e-4dee-befe-96cf93ba1372 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:48.218336 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8f2c1d-0bbd-4c01-818f-e9f0da446e46 service cinder] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:48.262758 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:00:48.321935 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2823264-641e-472e-9688-7a38b3d1d3c1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f4157105f3488f807b7475008ab722', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:48.322801 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2823264-641e-472e-9688-7a38b3d1d3c1 service neutron] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:48.323229 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2823264-641e-472e-9688-7a38b3d1d3c1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:48.323767 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2823264-641e-472e-9688-7a38b3d1d3c1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:48.344190 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2823264-641e-472e-9688-7a38b3d1d3c1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:48.346018 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2823264-641e-472e-9688-7a38b3d1d3c1 service neutron] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:48.350350 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-d929ba55-8f3e-4dee-befe-96cf93ba1372 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c0ddd4efe5344566a3b600b1c5e27120', 'request_id': 'req-d929ba55-8f3e-4dee-befe-96cf93ba1372'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:48.366526 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1053/2123] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 16:00:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 282 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 16:00:48.374516 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-d929ba55-8f3e-4dee-befe-96cf93ba1372 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c0ddd4efe5344566a3b600b1c5e27120', 'request_id': 'req-d929ba55-8f3e-4dee-befe-96cf93ba1372'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:48.417254 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-d929ba55-8f3e-4dee-befe-96cf93ba1372 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c0ddd4efe5344566a3b600b1c5e27120', 'request_id': 'req-d929ba55-8f3e-4dee-befe-96cf93ba1372'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:48.427204 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:00:48.445301 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-d929ba55-8f3e-4dee-befe-96cf93ba1372 admin admin] The token cache is being invalidate because project c0ddd4efe5344566a3b600b1c5e27120 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81511) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 16:00:48.445600 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-d929ba55-8f3e-4dee-befe-96cf93ba1372 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:48.446287 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-d929ba55-8f3e-4dee-befe-96cf93ba1372 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:48.452984 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1071/2124] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Aug 30 16:00:48 2022] DELETE /identity/v3/projects/c0ddd4efe5344566a3b600b1c5e27120 => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 16:00:48.458863 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0352285-b8c0-4388-aec2-f15c903a3851 None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:48.458951 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d929ba55-8f3e-4dee-befe-96cf93ba1372 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:48.459024 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0352285-b8c0-4388-aec2-f15c903a3851 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:48.459304 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0352285-b8c0-4388-aec2-f15c903a3851 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:48.460460 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-d929ba55-8f3e-4dee-befe-96cf93ba1372 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:00:48.530469 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-c0352285-b8c0-4388-aec2-f15c903a3851 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:00:48.534105 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-c0352285-b8c0-4388-aec2-f15c903a3851 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:00:48.592485 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d929ba55-8f3e-4dee-befe-96cf93ba1372 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:48.597565 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291e2639-a6d6-437d-a6d0-54daa5adce35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:48.598594 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291e2639-a6d6-437d-a6d0-54daa5adce35 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:48.598953 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291e2639-a6d6-437d-a6d0-54daa5adce35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:48.599942 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291e2639-a6d6-437d-a6d0-54daa5adce35 admin admin] PATH_INFO: `/v3/projects/cf03b164b1b443ada592eab5e9d2ef24` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:48.602548 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291e2639-a6d6-437d-a6d0-54daa5adce35 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cf03b164b1b443ada592eab5e9d2ef24)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:48.605392 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291e2639-a6d6-437d-a6d0-54daa5adce35 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:48.666081 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1054/2125] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 16:00:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:00:48.685735 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-291e2639-a6d6-437d-a6d0-54daa5adce35 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cf03b164b1b443ada592eab5e9d2ef24', 'request_id': 'req-291e2639-a6d6-437d-a6d0-54daa5adce35'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:48.687051 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd20a33f-c2ef-406d-b8f5-528e06dd74a6 None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:48.689587 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd20a33f-c2ef-406d-b8f5-528e06dd74a6 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:48.689951 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd20a33f-c2ef-406d-b8f5-528e06dd74a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:48.704521 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-291e2639-a6d6-437d-a6d0-54daa5adce35 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cf03b164b1b443ada592eab5e9d2ef24', 'request_id': 'req-291e2639-a6d6-437d-a6d0-54daa5adce35'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:48.716031 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-291e2639-a6d6-437d-a6d0-54daa5adce35 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cf03b164b1b443ada592eab5e9d2ef24', 'request_id': 'req-291e2639-a6d6-437d-a6d0-54daa5adce35'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:48.729150 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-291e2639-a6d6-437d-a6d0-54daa5adce35 admin admin] The token cache is being invalidate because project cf03b164b1b443ada592eab5e9d2ef24 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81511) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 16:00:48.729551 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-291e2639-a6d6-437d-a6d0-54daa5adce35 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:48.730240 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-291e2639-a6d6-437d-a6d0-54daa5adce35 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:48.732068 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1072/2126] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Aug 30 16:00:48 2022] DELETE /identity/v3/projects/cf03b164b1b443ada592eab5e9d2ef24 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 16:00:48.735746 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-cd20a33f-c2ef-406d-b8f5-528e06dd74a6 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:00:48.738198 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291e2639-a6d6-437d-a6d0-54daa5adce35 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:48.739305 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-291e2639-a6d6-437d-a6d0-54daa5adce35 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:00:48.739654 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-cd20a33f-c2ef-406d-b8f5-528e06dd74a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:00:48.839334 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291e2639-a6d6-437d-a6d0-54daa5adce35 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:48.846889 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7016067a-dddb-4613-80b3-93157612ada8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:48.848103 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7016067a-dddb-4613-80b3-93157612ada8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:48.848524 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7016067a-dddb-4613-80b3-93157612ada8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:48.848799 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7016067a-dddb-4613-80b3-93157612ada8 admin admin] PATH_INFO: `/v3/users/701d11ac33764ad78aa321cfdc6f5528` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:48.850007 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7016067a-dddb-4613-80b3-93157612ada8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=701d11ac33764ad78aa321cfdc6f5528)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:48.851939 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7016067a-dddb-4613-80b3-93157612ada8 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:48.854092 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1055/2127] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 16:00:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:00:48.858907 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd20a33f-c2ef-406d-b8f5-528e06dd74a6 None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:48.879238 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd20a33f-c2ef-406d-b8f5-528e06dd74a6 None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:48.884832 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed320f9e-76f7-4ec8-891f-8e3defa8d32a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:48.885580 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed320f9e-76f7-4ec8-891f-8e3defa8d32a admin admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:48.885863 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed320f9e-76f7-4ec8-891f-8e3defa8d32a admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:48.886150 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed320f9e-76f7-4ec8-891f-8e3defa8d32a admin admin] PATH_INFO: `/v3/domains` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:48.887031 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed320f9e-76f7-4ec8-891f-8e3defa8d32a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81512) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:00:48.887433 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed320f9e-76f7-4ec8-891f-8e3defa8d32a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:48.889278 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed320f9e-76f7-4ec8-891f-8e3defa8d32a admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:48.891865 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:00:48.904911 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1056/2128] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Aug 30 16:00:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:00:48.910039 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed320f9e-76f7-4ec8-891f-8e3defa8d32a admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:48.922967 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed320f9e-76f7-4ec8-891f-8e3defa8d32a admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:48.930241 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e1e120-91fa-4e3b-8a06-3e97c32b0746 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:48.931578 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e1e120-91fa-4e3b-8a06-3e97c32b0746 admin admin] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:48.931873 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e1e120-91fa-4e3b-8a06-3e97c32b0746 admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:48.932214 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e1e120-91fa-4e3b-8a06-3e97c32b0746 admin admin] PATH_INFO: `/v3/projects` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:48.933236 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e1e120-91fa-4e3b-8a06-3e97c32b0746 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:48.935389 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e1e120-91fa-4e3b-8a06-3e97c32b0746 admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:48.937144 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-7016067a-dddb-4613-80b3-93157612ada8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '701d11ac33764ad78aa321cfdc6f5528'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:48.965593 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-7016067a-dddb-4613-80b3-93157612ada8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '701d11ac33764ad78aa321cfdc6f5528'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:48.978959 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-7016067a-dddb-4613-80b3-93157612ada8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '701d11ac33764ad78aa321cfdc6f5528'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:48.979885 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-7016067a-dddb-4613-80b3-93157612ada8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '701d11ac33764ad78aa321cfdc6f5528'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:48.991500 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1057/2129] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Aug 30 16:00:48 2022] POST /identity/v3/projects => generated 357 bytes in 80 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 16:00:48.991500 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e1e120-91fa-4e3b-8a06-3e97c32b0746 admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:48.992837 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-25e1e120-91fa-4e3b-8a06-3e97c32b0746 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:00:48.995324 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-7016067a-dddb-4613-80b3-93157612ada8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '701d11ac33764ad78aa321cfdc6f5528'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:49.003746 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1073/2130] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Aug 30 16:00:48 2022] DELETE /identity/v3/users/701d11ac33764ad78aa321cfdc6f5528 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:00:49.004741 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7016067a-dddb-4613-80b3-93157612ada8 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:49.006449 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-7016067a-dddb-4613-80b3-93157612ada8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:00:49.074031 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:00:49.085024 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7016067a-dddb-4613-80b3-93157612ada8 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:49.092809 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf118419-70b4-4b7c-bb86-8d50d9c5e8f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:49.093609 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf118419-70b4-4b7c-bb86-8d50d9c5e8f2 admin admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:49.094626 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf118419-70b4-4b7c-bb86-8d50d9c5e8f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:49.095106 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf118419-70b4-4b7c-bb86-8d50d9c5e8f2 admin admin] PATH_INFO: `/v3/users` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:49.095835 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf118419-70b4-4b7c-bb86-8d50d9c5e8f2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:49.097918 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf118419-70b4-4b7c-bb86-8d50d9c5e8f2 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:49.118718 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e1e120-91fa-4e3b-8a06-3e97c32b0746 admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:49.124132 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a3392c-889f-44dd-936e-762b17e062db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:49.125094 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a3392c-889f-44dd-936e-762b17e062db admin admin] REQUEST_METHOD: `DELETE` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:49.129038 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1074/2131] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Aug 30 16:00:49 2022] POST /identity/v3/users => generated 279 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:00:49.129264 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a3392c-889f-44dd-936e-762b17e062db admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:49.129264 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a3392c-889f-44dd-936e-762b17e062db admin admin] PATH_INFO: `/v3/projects/febac92c68d64347bf169aecd6633f7d` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:49.129264 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a3392c-889f-44dd-936e-762b17e062db admin admin] RBAC: Authorizing `identity:delete_project(project_id=febac92c68d64347bf169aecd6633f7d)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:49.129774 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf118419-70b4-4b7c-bb86-8d50d9c5e8f2 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:49.130225 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a3392c-889f-44dd-936e-762b17e062db admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:49.140046 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf118419-70b4-4b7c-bb86-8d50d9c5e8f2 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:49.145936 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0580ba-3638-468d-8041-22e228c14b45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:49.146850 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0580ba-3638-468d-8041-22e228c14b45 admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:49.147126 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0580ba-3638-468d-8041-22e228c14b45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:49.147388 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0580ba-3638-468d-8041-22e228c14b45 admin admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:49.148030 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0580ba-3638-468d-8041-22e228c14b45 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:49.149688 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0580ba-3638-468d-8041-22e228c14b45 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:49.161567 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1075/2132] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 30 16:00:49 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 16:00:49.164643 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0580ba-3638-468d-8041-22e228c14b45 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:49.176166 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0580ba-3638-468d-8041-22e228c14b45 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:49.189858 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-f5a3392c-889f-44dd-936e-762b17e062db admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'febac92c68d64347bf169aecd6633f7d', 'request_id': 'req-f5a3392c-889f-44dd-936e-762b17e062db'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:49.200788 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-f5a3392c-889f-44dd-936e-762b17e062db admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'febac92c68d64347bf169aecd6633f7d', 'request_id': 'req-f5a3392c-889f-44dd-936e-762b17e062db'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:49.208536 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-f5a3392c-889f-44dd-936e-762b17e062db admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'febac92c68d64347bf169aecd6633f7d', 'request_id': 'req-f5a3392c-889f-44dd-936e-762b17e062db'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:49.219821 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-f5a3392c-889f-44dd-936e-762b17e062db admin admin] The token cache is being invalidate because project febac92c68d64347bf169aecd6633f7d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81512) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 16:00:49.220687 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-f5a3392c-889f-44dd-936e-762b17e062db admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:49.221583 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-f5a3392c-889f-44dd-936e-762b17e062db admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:49.223309 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1058/2133] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Aug 30 16:00:48 2022] DELETE /identity/v3/projects/febac92c68d64347bf169aecd6633f7d => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:00:49.228069 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421bd8db-2c3e-4dc5-808e-c17fcf7cd531 None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:49.228392 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421bd8db-2c3e-4dc5-808e-c17fcf7cd531 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:49.228652 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421bd8db-2c3e-4dc5-808e-c17fcf7cd531 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:49.250587 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c084591-451e-4ee5-9a4f-580e7ec04637 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:49.251289 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-421bd8db-2c3e-4dc5-808e-c17fcf7cd531 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:00:49.251396 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c084591-451e-4ee5-9a4f-580e7ec04637 admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:49.251499 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c084591-451e-4ee5-9a4f-580e7ec04637 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:49.251736 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c084591-451e-4ee5-9a4f-580e7ec04637 admin admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:49.252400 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c084591-451e-4ee5-9a4f-580e7ec04637 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:49.252498 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-421bd8db-2c3e-4dc5-808e-c17fcf7cd531 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:00:49.254073 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c084591-451e-4ee5-9a4f-580e7ec04637 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:49.261135 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1059/2134] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 16:00:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 16:00:49.265124 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1076/2135] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 30 16:00:49 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 16:00:49.269734 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421bd8db-2c3e-4dc5-808e-c17fcf7cd531 None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:49.270248 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb9a1e2-5c95-49cf-b74b-fd00b6ba334e None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:49.270439 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb9a1e2-5c95-49cf-b74b-fd00b6ba334e None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:49.270716 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb9a1e2-5c95-49cf-b74b-fd00b6ba334e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:49.271475 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-421bd8db-2c3e-4dc5-808e-c17fcf7cd531 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:00:49.285013 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421bd8db-2c3e-4dc5-808e-c17fcf7cd531 None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:49.290520 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f89e58-a567-4ef8-829b-3b382c0ebe4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:49.291180 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f89e58-a567-4ef8-829b-3b382c0ebe4b admin admin] REQUEST_METHOD: `PUT` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:49.291460 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f89e58-a567-4ef8-829b-3b382c0ebe4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:49.291680 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-6cb9a1e2-5c95-49cf-b74b-fd00b6ba334e None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:00:49.291864 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f89e58-a567-4ef8-829b-3b382c0ebe4b admin admin] PATH_INFO: `/v3/projects/310e0bee3254457db6cfb45c50e82cf6/users/bf02d15ca65a46fda47363c87eeaceaf/roles/df8f9f844b1b439181d75a07c1a93418` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:49.293306 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-6cb9a1e2-5c95-49cf-b74b-fd00b6ba334e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:00:49.302026 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1077/2136] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 16:00:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:00:49.305925 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb9a1e2-5c95-49cf-b74b-fd00b6ba334e None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:49.312218 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f89e58-a567-4ef8-829b-3b382c0ebe4b admin admin] RBAC: Authorizing `identity:create_grant(project_id=310e0bee3254457db6cfb45c50e82cf6, user_id=bf02d15ca65a46fda47363c87eeaceaf, role_id=df8f9f844b1b439181d75a07c1a93418)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:49.315436 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f89e58-a567-4ef8-829b-3b382c0ebe4b admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:49.315945 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb9a1e2-5c95-49cf-b74b-fd00b6ba334e None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:49.320730 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92b52a8-96ba-428c-869e-a8307b61ba3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:49.321457 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92b52a8-96ba-428c-869e-a8307b61ba3c admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:49.321793 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92b52a8-96ba-428c-869e-a8307b61ba3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:49.322007 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92b52a8-96ba-428c-869e-a8307b61ba3c admin admin] PATH_INFO: `/v3/domains` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:49.322664 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92b52a8-96ba-428c-869e-a8307b61ba3c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:00:49.322969 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92b52a8-96ba-428c-869e-a8307b61ba3c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:49.324677 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92b52a8-96ba-428c-869e-a8307b61ba3c admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:49.331486 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1060/2137] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 30 16:00:49 2022] PUT /identity/v3/projects/310e0bee3254457db6cfb45c50e82cf6/users/bf02d15ca65a46fda47363c87eeaceaf/roles/df8f9f844b1b439181d75a07c1a93418 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:00:49.341507 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:00:49.344669 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5447ca13-e60d-447d-8a9f-6e5a205f3399 None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:49.345139 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5447ca13-e60d-447d-8a9f-6e5a205f3399 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:49.345602 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5447ca13-e60d-447d-8a9f-6e5a205f3399 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:49.348199 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1078/2138] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Aug 30 16:00:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 16:00:49.352101 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92b52a8-96ba-428c-869e-a8307b61ba3c admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:49.364594 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92b52a8-96ba-428c-869e-a8307b61ba3c admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:49.385361 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-5447ca13-e60d-447d-8a9f-6e5a205f3399 None None] MFA Rules not processed for user `bf02d15ca65a46fda47363c87eeaceaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:00:49.386717 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-5447ca13-e60d-447d-8a9f-6e5a205f3399 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:00:49.432453 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42a213e-2761-49d0-86b4-6dbdd92ea1bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:49.433234 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42a213e-2761-49d0-86b4-6dbdd92ea1bb admin admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:49.433531 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42a213e-2761-49d0-86b4-6dbdd92ea1bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:49.433801 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42a213e-2761-49d0-86b4-6dbdd92ea1bb admin admin] PATH_INFO: `/v3/projects` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:49.434472 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42a213e-2761-49d0-86b4-6dbdd92ea1bb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:49.436847 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42a213e-2761-49d0-86b4-6dbdd92ea1bb admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:49.468714 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1061/2139] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 16:00:49 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:00:49.470127 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5447ca13-e60d-447d-8a9f-6e5a205f3399 None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:49.471871 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-5447ca13-e60d-447d-8a9f-6e5a205f3399 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:00:49.477067 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1079/2140] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Aug 30 16:00:49 2022] POST /identity/v3/projects => generated 385 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 16:00:49.482111 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42a213e-2761-49d0-86b4-6dbdd92ea1bb admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:49.490472 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5447ca13-e60d-447d-8a9f-6e5a205f3399 None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:49.513590 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:00:49.570733 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42a213e-2761-49d0-86b4-6dbdd92ea1bb admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:49.584999 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881b07e4-db01-482d-8df4-baa63f66de52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:49.585967 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881b07e4-db01-482d-8df4-baa63f66de52 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:49.586312 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881b07e4-db01-482d-8df4-baa63f66de52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:49.586694 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881b07e4-db01-482d-8df4-baa63f66de52 admin admin] PATH_INFO: `/v3/projects/7404694160dd43478d60906173c3a16f` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:49.588360 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a599709-f38e-4a30-881b-37612141ea39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:49.588506 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881b07e4-db01-482d-8df4-baa63f66de52 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7404694160dd43478d60906173c3a16f)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:49.589730 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a599709-f38e-4a30-881b-37612141ea39 admin admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:49.590406 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a599709-f38e-4a30-881b-37612141ea39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:49.591293 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881b07e4-db01-482d-8df4-baa63f66de52 admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:49.591599 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a599709-f38e-4a30-881b-37612141ea39 admin admin] PATH_INFO: `/v3/projects` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:49.592540 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a599709-f38e-4a30-881b-37612141ea39 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:49.594843 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a599709-f38e-4a30-881b-37612141ea39 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:49.635324 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1080/2141] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Aug 30 16:00:49 2022] POST /identity/v3/projects => generated 355 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:00:49.636315 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a599709-f38e-4a30-881b-37612141ea39 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:49.675666 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-881b07e4-db01-482d-8df4-baa63f66de52 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7404694160dd43478d60906173c3a16f', 'request_id': 'req-881b07e4-db01-482d-8df4-baa63f66de52'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:49.686147 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-881b07e4-db01-482d-8df4-baa63f66de52 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7404694160dd43478d60906173c3a16f', 'request_id': 'req-881b07e4-db01-482d-8df4-baa63f66de52'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:49.695972 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-881b07e4-db01-482d-8df4-baa63f66de52 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7404694160dd43478d60906173c3a16f', 'request_id': 'req-881b07e4-db01-482d-8df4-baa63f66de52'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:49.708241 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-881b07e4-db01-482d-8df4-baa63f66de52 admin admin] The token cache is being invalidate because project 7404694160dd43478d60906173c3a16f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81512) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 16:00:49.708798 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-881b07e4-db01-482d-8df4-baa63f66de52 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:49.709772 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-881b07e4-db01-482d-8df4-baa63f66de52 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:49.712003 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1062/2142] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Aug 30 16:00:49 2022] DELETE /identity/v3/projects/7404694160dd43478d60906173c3a16f => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:00:49.713581 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881b07e4-db01-482d-8df4-baa63f66de52 admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:49.715182 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-881b07e4-db01-482d-8df4-baa63f66de52 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:00:49.717552 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a599709-f38e-4a30-881b-37612141ea39 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:49.730884 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:00:49.793780 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168478c6-8824-4088-aaca-bd8886ba1154 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:49.794540 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168478c6-8824-4088-aaca-bd8886ba1154 admin admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:49.795318 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168478c6-8824-4088-aaca-bd8886ba1154 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:49.795318 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168478c6-8824-4088-aaca-bd8886ba1154 admin admin] PATH_INFO: `/v3/users` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:49.795629 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881b07e4-db01-482d-8df4-baa63f66de52 admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:49.795710 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168478c6-8824-4088-aaca-bd8886ba1154 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:49.797851 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168478c6-8824-4088-aaca-bd8886ba1154 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:49.800496 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2103e787-373f-4602-91ba-b64d27c723fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:49.801108 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2103e787-373f-4602-91ba-b64d27c723fb admin admin] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:49.801962 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2103e787-373f-4602-91ba-b64d27c723fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:49.801962 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2103e787-373f-4602-91ba-b64d27c723fb admin admin] PATH_INFO: `/v3/users` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:49.802423 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2103e787-373f-4602-91ba-b64d27c723fb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:49.804485 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2103e787-373f-4602-91ba-b64d27c723fb admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:49.824083 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1081/2143] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Aug 30 16:00:49 2022] POST /identity/v3/users => generated 293 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:00:49.825743 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dafbd4-2c55-4778-9769-a24edd857330 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:49.825876 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dafbd4-2c55-4778-9769-a24edd857330 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:49.826057 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dafbd4-2c55-4778-9769-a24edd857330 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:49.831654 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1063/2144] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Aug 30 16:00:49 2022] POST /identity/v3/users => generated 284 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:00:49.832591 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2103e787-373f-4602-91ba-b64d27c723fb admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:49.835536 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-2103e787-373f-4602-91ba-b64d27c723fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:00:49.850706 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2103e787-373f-4602-91ba-b64d27c723fb admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:49.854922 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-a2dafbd4-2c55-4778-9769-a24edd857330 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:00:49.856284 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-a2dafbd4-2c55-4778-9769-a24edd857330 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:00:49.858146 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2761d1-709b-40cf-a447-b0adecde3f4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:49.858792 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2761d1-709b-40cf-a447-b0adecde3f4f admin admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:49.859073 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2761d1-709b-40cf-a447-b0adecde3f4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:49.859431 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2761d1-709b-40cf-a447-b0adecde3f4f admin admin] PATH_INFO: `/v3/roles` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:49.860295 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2761d1-709b-40cf-a447-b0adecde3f4f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:49.862176 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2761d1-709b-40cf-a447-b0adecde3f4f admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:49.865637 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1082/2145] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 16:00:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:00:49.866372 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2dafbd4-2c55-4778-9769-a24edd857330 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:49.873458 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1064/2146] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 30 16:00:49 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:00:49.874498 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c380a69-80d4-4721-b949-60f6af17d8c5 None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:49.874796 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c380a69-80d4-4721-b949-60f6af17d8c5 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:49.875088 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c380a69-80d4-4721-b949-60f6af17d8c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:49.882278 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2dafbd4-2c55-4778-9769-a24edd857330 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:49.887992 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9ed9f8-85a1-42b1-8642-7ed27198ed21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:49.888656 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9ed9f8-85a1-42b1-8642-7ed27198ed21 admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:49.888940 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9ed9f8-85a1-42b1-8642-7ed27198ed21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:49.889211 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9ed9f8-85a1-42b1-8642-7ed27198ed21 admin admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:49.889896 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9ed9f8-85a1-42b1-8642-7ed27198ed21 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:49.892030 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9ed9f8-85a1-42b1-8642-7ed27198ed21 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:49.895807 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-3c380a69-80d4-4721-b949-60f6af17d8c5 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:00:49.897042 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-3c380a69-80d4-4721-b949-60f6af17d8c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:00:49.902494 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1083/2147] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 30 16:00:49 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:00:49.903305 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9ed9f8-85a1-42b1-8642-7ed27198ed21 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:49.905941 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1065/2148] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 16:00:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:00:49.909475 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c380a69-80d4-4721-b949-60f6af17d8c5 None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:49.916289 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9ed9f8-85a1-42b1-8642-7ed27198ed21 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:49.918586 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c380a69-80d4-4721-b949-60f6af17d8c5 None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:49.921481 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1fe4ad-b5d4-446e-8ecf-a8e1edb6f126 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:49.922185 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1fe4ad-b5d4-446e-8ecf-a8e1edb6f126 admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:49.922488 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1fe4ad-b5d4-446e-8ecf-a8e1edb6f126 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:49.922753 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1fe4ad-b5d4-446e-8ecf-a8e1edb6f126 admin admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:49.923427 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1fe4ad-b5d4-446e-8ecf-a8e1edb6f126 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:49.923741 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836bf5d2-7e10-4596-9dd9-d0be4ba5f6e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:49.924396 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836bf5d2-7e10-4596-9dd9-d0be4ba5f6e3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:49.924690 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836bf5d2-7e10-4596-9dd9-d0be4ba5f6e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:49.924946 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836bf5d2-7e10-4596-9dd9-d0be4ba5f6e3 admin admin] PATH_INFO: `/v3/projects/be93ec67199443579907c5d948e925b8/users/277631c6432e4ad0a5f98f9052301fd7/roles/f265386950a641ac8413b53eda4ae34d` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:49.925036 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1fe4ad-b5d4-446e-8ecf-a8e1edb6f126 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:49.940040 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1084/2149] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 30 16:00:49 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:00:49.940727 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1fe4ad-b5d4-446e-8ecf-a8e1edb6f126 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:49.943174 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836bf5d2-7e10-4596-9dd9-d0be4ba5f6e3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=be93ec67199443579907c5d948e925b8, user_id=277631c6432e4ad0a5f98f9052301fd7, role_id=f265386950a641ac8413b53eda4ae34d)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:49.945933 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836bf5d2-7e10-4596-9dd9-d0be4ba5f6e3 admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:49.950581 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1fe4ad-b5d4-446e-8ecf-a8e1edb6f126 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:49.955541 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1e9df2-3979-4057-bb5b-dfb77cd09e86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:49.956020 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:00:49.956266 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1e9df2-3979-4057-bb5b-dfb77cd09e86 admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:49.956602 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1e9df2-3979-4057-bb5b-dfb77cd09e86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:49.956856 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1e9df2-3979-4057-bb5b-dfb77cd09e86 admin admin] PATH_INFO: `/v3/domains` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:49.957563 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1e9df2-3979-4057-bb5b-dfb77cd09e86 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:00:49.957860 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1e9df2-3979-4057-bb5b-dfb77cd09e86 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:49.959571 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1e9df2-3979-4057-bb5b-dfb77cd09e86 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:49.962024 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1066/2150] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 30 16:00:49 2022] PUT /identity/v3/projects/be93ec67199443579907c5d948e925b8/users/277631c6432e4ad0a5f98f9052301fd7/roles/f265386950a641ac8413b53eda4ae34d => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:00:49.963358 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836bf5d2-7e10-4596-9dd9-d0be4ba5f6e3 admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:49.978518 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1085/2151] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Aug 30 16:00:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:00:49.978518 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1e9df2-3979-4057-bb5b-dfb77cd09e86 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:49.986203 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1e9df2-3979-4057-bb5b-dfb77cd09e86 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:50.029906 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836bf5d2-7e10-4596-9dd9-d0be4ba5f6e3 admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:50.034678 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fbd134-ec4b-4dda-af1f-7515a8290221 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:50.035346 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fbd134-ec4b-4dda-af1f-7515a8290221 admin admin] REQUEST_METHOD: `PUT` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:50.035624 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fbd134-ec4b-4dda-af1f-7515a8290221 admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:50.035898 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fbd134-ec4b-4dda-af1f-7515a8290221 admin admin] PATH_INFO: `/v3/projects/26bb6b7007af4fdda66f640d3f258927/users/848e370c25d04e70b04f6754a70ffa28/roles/df8f9f844b1b439181d75a07c1a93418` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:50.042920 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05be3c0a-98c4-495a-9f91-38e36d3e701d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:50.043527 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05be3c0a-98c4-495a-9f91-38e36d3e701d admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:50.043805 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05be3c0a-98c4-495a-9f91-38e36d3e701d admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:50.044074 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05be3c0a-98c4-495a-9f91-38e36d3e701d admin admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:50.044742 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05be3c0a-98c4-495a-9f91-38e36d3e701d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:50.046430 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05be3c0a-98c4-495a-9f91-38e36d3e701d admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:50.050462 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6fbd134-ec4b-4dda-af1f-7515a8290221 admin admin] RBAC: Authorizing `identity:create_grant(project_id=26bb6b7007af4fdda66f640d3f258927, user_id=848e370c25d04e70b04f6754a70ffa28, role_id=df8f9f844b1b439181d75a07c1a93418)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:50.053117 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6fbd134-ec4b-4dda-af1f-7515a8290221 admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:50.056929 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1086/2152] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 30 16:00:49 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:00:50.057809 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05be3c0a-98c4-495a-9f91-38e36d3e701d admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:50.068014 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05be3c0a-98c4-495a-9f91-38e36d3e701d admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:50.073484 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1067/2153] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 30 16:00:49 2022] PUT /identity/v3/projects/26bb6b7007af4fdda66f640d3f258927/users/848e370c25d04e70b04f6754a70ffa28/roles/df8f9f844b1b439181d75a07c1a93418 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:00:50.074244 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fbd134-ec4b-4dda-af1f-7515a8290221 admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:50.146335 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309e2224-97a8-4cf1-ab43-575f67659e80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:50.147333 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309e2224-97a8-4cf1-ab43-575f67659e80 admin admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:50.147811 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309e2224-97a8-4cf1-ab43-575f67659e80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:50.148318 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309e2224-97a8-4cf1-ab43-575f67659e80 admin admin] PATH_INFO: `/v3/projects` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:50.149787 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309e2224-97a8-4cf1-ab43-575f67659e80 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:50.153136 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309e2224-97a8-4cf1-ab43-575f67659e80 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:50.156630 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fbd134-ec4b-4dda-af1f-7515a8290221 admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:50.158926 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:00:50.162021 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefc5f63-90a1-40c3-af33-a04d916ef198 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:50.162772 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefc5f63-90a1-40c3-af33-a04d916ef198 admin admin] REQUEST_METHOD: `PUT` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:50.163082 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefc5f63-90a1-40c3-af33-a04d916ef198 admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:50.163356 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefc5f63-90a1-40c3-af33-a04d916ef198 admin admin] PATH_INFO: `/v3/projects/be93ec67199443579907c5d948e925b8/users/277631c6432e4ad0a5f98f9052301fd7/roles/f265386950a641ac8413b53eda4ae34d` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:50.165647 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fefc5f63-90a1-40c3-af33-a04d916ef198 admin admin] RBAC: Authorizing `identity:create_grant(project_id=be93ec67199443579907c5d948e925b8, user_id=277631c6432e4ad0a5f98f9052301fd7, role_id=f265386950a641ac8413b53eda4ae34d)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:50.168650 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fefc5f63-90a1-40c3-af33-a04d916ef198 admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:50.182622 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1068/2154] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 30 16:00:50 2022] PUT /identity/v3/projects/be93ec67199443579907c5d948e925b8/users/277631c6432e4ad0a5f98f9052301fd7/roles/f265386950a641ac8413b53eda4ae34d => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:00:50.184751 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e4d97e-4670-4e47-86b8-1940f4b60d80 None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:50.194649 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e4d97e-4670-4e47-86b8-1940f4b60d80 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:50.194649 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e4d97e-4670-4e47-86b8-1940f4b60d80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:50.208219 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1087/2155] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Aug 30 16:00:50 2022] POST /identity/v3/projects => generated 379 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:00:50.208219 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c9dd2e-d67a-41e6-9edc-90c9ee282c23 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:50.208219 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c9dd2e-d67a-41e6-9edc-90c9ee282c23 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:50.208219 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c9dd2e-d67a-41e6-9edc-90c9ee282c23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:50.228773 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-93e4d97e-4670-4e47-86b8-1940f4b60d80 None None] MFA Rules not processed for user `848e370c25d04e70b04f6754a70ffa28`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:00:50.231516 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-93e4d97e-4670-4e47-86b8-1940f4b60d80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:00:50.242678 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-b8c9dd2e-d67a-41e6-9edc-90c9ee282c23 None None] MFA Rules not processed for user `277631c6432e4ad0a5f98f9052301fd7`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:00:50.244238 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-b8c9dd2e-d67a-41e6-9edc-90c9ee282c23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:00:50.304119 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1069/2156] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 16:00:50 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:00:50.306114 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e4d97e-4670-4e47-86b8-1940f4b60d80 None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:50.314638 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:00:50.315321 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1088/2157] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 16:00:50 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:00:50.318266 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c9dd2e-d67a-41e6-9edc-90c9ee282c23 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:50.378430 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e4d97e-4670-4e47-86b8-1940f4b60d80 None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:50.387690 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e91a092-93dc-4ae6-aa4e-34212085bbd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:50.388302 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e91a092-93dc-4ae6-aa4e-34212085bbd8 admin admin] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:50.388587 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e91a092-93dc-4ae6-aa4e-34212085bbd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:50.388755 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c9dd2e-d67a-41e6-9edc-90c9ee282c23 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:50.388874 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e91a092-93dc-4ae6-aa4e-34212085bbd8 admin admin] PATH_INFO: `/v3/users` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:50.389515 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e91a092-93dc-4ae6-aa4e-34212085bbd8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:50.391561 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e91a092-93dc-4ae6-aa4e-34212085bbd8 admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:50.394202 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4272415-86c1-419d-94cc-86e79535ee06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:50.394862 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4272415-86c1-419d-94cc-86e79535ee06 admin admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:50.395076 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4272415-86c1-419d-94cc-86e79535ee06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:50.395469 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4272415-86c1-419d-94cc-86e79535ee06 admin admin] PATH_INFO: `/v3/projects` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:50.397158 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4272415-86c1-419d-94cc-86e79535ee06 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:50.399139 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4272415-86c1-419d-94cc-86e79535ee06 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:50.419690 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1070/2158] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Aug 30 16:00:50 2022] POST /identity/v3/users => generated 290 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:00:50.420716 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e91a092-93dc-4ae6-aa4e-34212085bbd8 admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:50.422616 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-8e91a092-93dc-4ae6-aa4e-34212085bbd8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:00:50.435987 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1089/2159] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Aug 30 16:00:50 2022] POST /identity/v3/projects => generated 385 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:00:50.436808 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4272415-86c1-419d-94cc-86e79535ee06 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:50.480722 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e91a092-93dc-4ae6-aa4e-34212085bbd8 admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:50.495989 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:00:50.501091 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4272415-86c1-419d-94cc-86e79535ee06 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:50.507300 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a86fee6-094f-49b2-8e8d-7d511828ec2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:50.507913 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a86fee6-094f-49b2-8e8d-7d511828ec2b admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:50.508222 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a86fee6-094f-49b2-8e8d-7d511828ec2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:50.508487 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a86fee6-094f-49b2-8e8d-7d511828ec2b admin admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:50.509134 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a86fee6-094f-49b2-8e8d-7d511828ec2b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:50.510794 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a86fee6-094f-49b2-8e8d-7d511828ec2b admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:50.520991 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1090/2160] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 30 16:00:50 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:00:50.522017 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a86fee6-094f-49b2-8e8d-7d511828ec2b admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:50.531468 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a86fee6-094f-49b2-8e8d-7d511828ec2b admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:50.533384 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1b75c9-431d-4754-b33d-a9e1ee3369d5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0100a04821804993866ea20f159663f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:50.533999 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1b75c9-431d-4754-b33d-a9e1ee3369d5 service cinder] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:50.534298 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1b75c9-431d-4754-b33d-a9e1ee3369d5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:50.534562 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1b75c9-431d-4754-b33d-a9e1ee3369d5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:50.536246 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1802ac96-98cf-4caa-9895-f4295cbfb400 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:50.536834 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1802ac96-98cf-4caa-9895-f4295cbfb400 admin admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:50.537138 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1802ac96-98cf-4caa-9895-f4295cbfb400 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:50.537434 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1802ac96-98cf-4caa-9895-f4295cbfb400 admin admin] PATH_INFO: `/v3/users` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:50.538065 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1802ac96-98cf-4caa-9895-f4295cbfb400 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:50.539962 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1802ac96-98cf-4caa-9895-f4295cbfb400 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:50.562702 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1091/2161] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Aug 30 16:00:50 2022] POST /identity/v3/users => generated 293 bytes in 41 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:00:50.564876 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1802ac96-98cf-4caa-9895-f4295cbfb400 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:50.574876 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1802ac96-98cf-4caa-9895-f4295cbfb400 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:50.579505 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0426e5e2-0634-4f58-bcf5-c407aa983364 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:50.580090 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0426e5e2-0634-4f58-bcf5-c407aa983364 admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:50.580364 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0426e5e2-0634-4f58-bcf5-c407aa983364 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:50.580628 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0426e5e2-0634-4f58-bcf5-c407aa983364 admin admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:50.581289 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0426e5e2-0634-4f58-bcf5-c407aa983364 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:50.582891 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0426e5e2-0634-4f58-bcf5-c407aa983364 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:50.600184 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1092/2162] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 30 16:00:50 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:00:50.601208 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0426e5e2-0634-4f58-bcf5-c407aa983364 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:50.604129 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1b75c9-431d-4754-b33d-a9e1ee3369d5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:50.605635 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1b75c9-431d-4754-b33d-a9e1ee3369d5 service cinder] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:50.612621 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0426e5e2-0634-4f58-bcf5-c407aa983364 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:50.616915 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1071/2163] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 16:00:50 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:00:50.617791 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1b75c9-431d-4754-b33d-a9e1ee3369d5 service cinder] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:50.618500 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944cb505-3b7b-4451-8364-6eda00b5ceb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:50.619288 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944cb505-3b7b-4451-8364-6eda00b5ceb8 admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:50.619571 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944cb505-3b7b-4451-8364-6eda00b5ceb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:50.619843 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944cb505-3b7b-4451-8364-6eda00b5ceb8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:50.620654 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944cb505-3b7b-4451-8364-6eda00b5ceb8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:50.622400 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944cb505-3b7b-4451-8364-6eda00b5ceb8 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:50.623548 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:00:50.632301 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1b75c9-431d-4754-b33d-a9e1ee3369d5 service cinder] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:50.638616 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb8cdf3-53b8-4b99-95e0-07334ccaeb10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:50.639715 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb8cdf3-53b8-4b99-95e0-07334ccaeb10 admin admin] REQUEST_METHOD: `PUT` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:50.640008 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb8cdf3-53b8-4b99-95e0-07334ccaeb10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:50.640281 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb8cdf3-53b8-4b99-95e0-07334ccaeb10 admin admin] PATH_INFO: `/v3/projects/0b57b04b591c413d8e9786b657833ac4/users/7a12b37a283740a090a45b5d76247cd1/roles/df8f9f844b1b439181d75a07c1a93418` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:50.645812 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1093/2164] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 30 16:00:50 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:00:50.648430 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944cb505-3b7b-4451-8364-6eda00b5ceb8 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:50.661326 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb8cdf3-53b8-4b99-95e0-07334ccaeb10 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b57b04b591c413d8e9786b657833ac4, user_id=7a12b37a283740a090a45b5d76247cd1, role_id=df8f9f844b1b439181d75a07c1a93418)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:50.663286 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944cb505-3b7b-4451-8364-6eda00b5ceb8 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:50.664260 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb8cdf3-53b8-4b99-95e0-07334ccaeb10 admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:50.671866 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8afe63-ac11-4578-b283-0030e6a36631 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:50.672933 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8afe63-ac11-4578-b283-0030e6a36631 admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:50.673434 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8afe63-ac11-4578-b283-0030e6a36631 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:50.673873 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8afe63-ac11-4578-b283-0030e6a36631 admin admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:50.680651 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8afe63-ac11-4578-b283-0030e6a36631 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:50.680651 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8afe63-ac11-4578-b283-0030e6a36631 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:50.689989 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1094/2165] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 30 16:00:50 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:00:50.692616 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1072/2166] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 30 16:00:50 2022] PUT /identity/v3/projects/0b57b04b591c413d8e9786b657833ac4/users/7a12b37a283740a090a45b5d76247cd1/roles/df8f9f844b1b439181d75a07c1a93418 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:00:50.704739 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea80f336-f46d-49c9-a993-f6f5559cb578 None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:50.705093 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8afe63-ac11-4578-b283-0030e6a36631 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:50.706525 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea80f336-f46d-49c9-a993-f6f5559cb578 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:50.706854 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea80f336-f46d-49c9-a993-f6f5559cb578 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:50.718719 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8afe63-ac11-4578-b283-0030e6a36631 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:50.751237 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-ea80f336-f46d-49c9-a993-f6f5559cb578 None None] MFA Rules not processed for user `7a12b37a283740a090a45b5d76247cd1`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:00:50.752964 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-ea80f336-f46d-49c9-a993-f6f5559cb578 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:00:50.789097 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01676f1c-66d0-4d95-9094-fa0499b10a8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:50.789097 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01676f1c-66d0-4d95-9094-fa0499b10a8d admin admin] REQUEST_METHOD: `PUT` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:50.789097 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01676f1c-66d0-4d95-9094-fa0499b10a8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:50.789097 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01676f1c-66d0-4d95-9094-fa0499b10a8d admin admin] PATH_INFO: `/v3/projects/8bd4ac278cba4b7c85bd0930cb03f7cb/users/2bfce39a1e534209a9dcdae3a2683c7f/roles/df8f9f844b1b439181d75a07c1a93418` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:50.801502 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01676f1c-66d0-4d95-9094-fa0499b10a8d admin admin] RBAC: Authorizing `identity:create_grant(project_id=8bd4ac278cba4b7c85bd0930cb03f7cb, user_id=2bfce39a1e534209a9dcdae3a2683c7f, role_id=df8f9f844b1b439181d75a07c1a93418)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:50.804164 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01676f1c-66d0-4d95-9094-fa0499b10a8d admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:50.821842 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1095/2167] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 30 16:00:50 2022] PUT /identity/v3/projects/8bd4ac278cba4b7c85bd0930cb03f7cb/users/2bfce39a1e534209a9dcdae3a2683c7f/roles/df8f9f844b1b439181d75a07c1a93418 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:00:50.823226 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01676f1c-66d0-4d95-9094-fa0499b10a8d admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:50.832002 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1073/2168] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 16:00:50 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:00:50.837253 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e805545-a2bc-4e5f-bb19-02df236a6620 None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:50.837490 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e805545-a2bc-4e5f-bb19-02df236a6620 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:50.837809 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e805545-a2bc-4e5f-bb19-02df236a6620 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:50.874355 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-4e805545-a2bc-4e5f-bb19-02df236a6620 None None] MFA Rules not processed for user `2bfce39a1e534209a9dcdae3a2683c7f`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:00:50.875622 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-4e805545-a2bc-4e5f-bb19-02df236a6620 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:00:50.900222 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01676f1c-66d0-4d95-9094-fa0499b10a8d admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:50.905153 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df1ab99-7a16-43ec-982a-c0a1788e87e6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0100a04821804993866ea20f159663f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:50.905803 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df1ab99-7a16-43ec-982a-c0a1788e87e6 service cinder] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:50.906081 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df1ab99-7a16-43ec-982a-c0a1788e87e6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:50.906358 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df1ab99-7a16-43ec-982a-c0a1788e87e6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:50.908538 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-6df1ab99-7a16-43ec-982a-c0a1788e87e6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:00:50.971747 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:00:50.978637 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1074/2169] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 16:00:50 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:00:50.979833 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e805545-a2bc-4e5f-bb19-02df236a6620 None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:51.002150 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df1ab99-7a16-43ec-982a-c0a1788e87e6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:51.003599 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df1ab99-7a16-43ec-982a-c0a1788e87e6 service cinder] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:51.012745 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1096/2170] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 16:00:50 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:00:51.014206 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf50ce9-0405-4b93-89f5-39f09ae9f379 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:51.014776 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf50ce9-0405-4b93-89f5-39f09ae9f379 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:51.014776 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf50ce9-0405-4b93-89f5-39f09ae9f379 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:51.041142 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-dcf50ce9-0405-4b93-89f5-39f09ae9f379 None None] MFA Rules not processed for user `07130d598131443486c2bc1c6673c45b`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:00:51.042411 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-dcf50ce9-0405-4b93-89f5-39f09ae9f379 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:00:51.051970 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:00:51.059118 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e805545-a2bc-4e5f-bb19-02df236a6620 None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:51.067691 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082f4547-b18d-425a-a467-14226445de7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:51.068390 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082f4547-b18d-425a-a467-14226445de7c admin admin] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:51.068714 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082f4547-b18d-425a-a467-14226445de7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:51.068987 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082f4547-b18d-425a-a467-14226445de7c admin admin] PATH_INFO: `/v3/projects` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:51.069745 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082f4547-b18d-425a-a467-14226445de7c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:51.072701 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082f4547-b18d-425a-a467-14226445de7c admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:51.122182 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1075/2171] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Aug 30 16:00:50 2022] POST /identity/v3/projects => generated 379 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:00:51.123174 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082f4547-b18d-425a-a467-14226445de7c admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:51.127452 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1097/2172] 10.222.0.20 () {58 vars in 1009 bytes} [Tue Aug 30 16:00:51 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:00:51.128599 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf50ce9-0405-4b93-89f5-39f09ae9f379 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:51.152452 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf50ce9-0405-4b93-89f5-39f09ae9f379 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:51.242260 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60343cdb-887a-4291-bfcb-84829b61f7ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:51.242882 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60343cdb-887a-4291-bfcb-84829b61f7ec admin admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:51.243218 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60343cdb-887a-4291-bfcb-84829b61f7ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:51.243495 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60343cdb-887a-4291-bfcb-84829b61f7ec admin admin] PATH_INFO: `/v3/users` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:51.244136 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60343cdb-887a-4291-bfcb-84829b61f7ec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:51.246122 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60343cdb-887a-4291-bfcb-84829b61f7ec admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:51.289124 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1098/2173] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Aug 30 16:00:51 2022] POST /identity/v3/users => generated 296 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:00:51.290419 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60343cdb-887a-4291-bfcb-84829b61f7ec admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:51.292407 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-60343cdb-887a-4291-bfcb-84829b61f7ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:00:51.304459 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082f4547-b18d-425a-a467-14226445de7c admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:51.310898 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74df5d56-4611-4234-a3b7-0171f1084ed3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:51.311597 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74df5d56-4611-4234-a3b7-0171f1084ed3 admin admin] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:51.311898 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74df5d56-4611-4234-a3b7-0171f1084ed3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:51.312181 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74df5d56-4611-4234-a3b7-0171f1084ed3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:51.312880 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74df5d56-4611-4234-a3b7-0171f1084ed3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:51.315262 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74df5d56-4611-4234-a3b7-0171f1084ed3 admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:51.356613 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1076/2174] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Aug 30 16:00:51 2022] POST /identity/v3/projects => generated 385 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:00:51.357729 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74df5d56-4611-4234-a3b7-0171f1084ed3 admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:51.395368 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60343cdb-887a-4291-bfcb-84829b61f7ec admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:51.412329 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:00:51.449917 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74df5d56-4611-4234-a3b7-0171f1084ed3 admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:51.477982 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71af0aff-b688-4a74-ad04-c2316d63dad7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:51.477982 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71af0aff-b688-4a74-ad04-c2316d63dad7 admin admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:51.477982 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71af0aff-b688-4a74-ad04-c2316d63dad7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:51.477982 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71af0aff-b688-4a74-ad04-c2316d63dad7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:51.478506 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71af0aff-b688-4a74-ad04-c2316d63dad7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:51.478506 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71af0aff-b688-4a74-ad04-c2316d63dad7 admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:51.486537 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbeff047-d67c-4df5-9b22-458d55aca85a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8fdafdc39a594bb4b0aeaf2fbc84c2fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:51.487253 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbeff047-d67c-4df5-9b22-458d55aca85a service swift] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:51.488047 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbeff047-d67c-4df5-9b22-458d55aca85a service swift] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:51.488300 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbeff047-d67c-4df5-9b22-458d55aca85a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:51.492572 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1077/2175] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 30 16:00:51 2022] GET /identity/v3/roles => generated 1420 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:00:51.493704 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71af0aff-b688-4a74-ad04-c2316d63dad7 admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:51.504355 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71af0aff-b688-4a74-ad04-c2316d63dad7 admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:51.508990 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5f5511-d38d-4571-97be-702bbf638c5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:51.509642 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5f5511-d38d-4571-97be-702bbf638c5a admin admin] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:51.510004 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5f5511-d38d-4571-97be-702bbf638c5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:51.510279 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5f5511-d38d-4571-97be-702bbf638c5a admin admin] PATH_INFO: `/v3/users` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:51.510947 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5f5511-d38d-4571-97be-702bbf638c5a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:51.512887 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5f5511-d38d-4571-97be-702bbf638c5a admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:51.540220 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1078/2176] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Aug 30 16:00:51 2022] POST /identity/v3/users => generated 299 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:00:51.543493 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5f5511-d38d-4571-97be-702bbf638c5a admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:51.552455 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbeff047-d67c-4df5-9b22-458d55aca85a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:51.554481 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5f5511-d38d-4571-97be-702bbf638c5a admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:51.554619 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbeff047-d67c-4df5-9b22-458d55aca85a service swift] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:51.559104 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6125bb-4838-4976-8d4c-4e53e4c74a04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:51.559836 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6125bb-4838-4976-8d4c-4e53e4c74a04 admin admin] REQUEST_METHOD: `PUT` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:51.560180 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6125bb-4838-4976-8d4c-4e53e4c74a04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:51.560538 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6125bb-4838-4976-8d4c-4e53e4c74a04 admin admin] PATH_INFO: `/v3/projects/81de2fd166dc4a02a439dbffd71d1bcd/users/ce10e46249a14592a1d27b3413a4b9fc/roles/f265386950a641ac8413b53eda4ae34d` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:51.563470 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1099/2177] 10.222.0.20 () {60 vars in 1384 bytes} [Tue Aug 30 16:00:51 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 273 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 30 16:00:51.564376 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbeff047-d67c-4df5-9b22-458d55aca85a service swift] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:51.580016 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6125bb-4838-4976-8d4c-4e53e4c74a04 admin admin] RBAC: Authorizing `identity:create_grant(project_id=81de2fd166dc4a02a439dbffd71d1bcd, user_id=ce10e46249a14592a1d27b3413a4b9fc, role_id=f265386950a641ac8413b53eda4ae34d)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:51.583159 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6125bb-4838-4976-8d4c-4e53e4c74a04 admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:51.587172 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbeff047-d67c-4df5-9b22-458d55aca85a service swift] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:51.593343 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a33953c-74ba-4266-a33e-d47205671afd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:51.594024 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a33953c-74ba-4266-a33e-d47205671afd admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:51.594428 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a33953c-74ba-4266-a33e-d47205671afd admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:51.594760 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a33953c-74ba-4266-a33e-d47205671afd admin admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:51.595466 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a33953c-74ba-4266-a33e-d47205671afd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:51.597270 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a33953c-74ba-4266-a33e-d47205671afd admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:51.601502 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1079/2178] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 30 16:00:51 2022] PUT /identity/v3/projects/81de2fd166dc4a02a439dbffd71d1bcd/users/ce10e46249a14592a1d27b3413a4b9fc/roles/f265386950a641ac8413b53eda4ae34d => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:00:51.605292 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6125bb-4838-4976-8d4c-4e53e4c74a04 admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:51.614097 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1100/2179] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 30 16:00:51 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:00:51.619327 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a33953c-74ba-4266-a33e-d47205671afd admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:51.631646 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a33953c-74ba-4266-a33e-d47205671afd admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:51.651559 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:00:51.718010 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6125bb-4838-4976-8d4c-4e53e4c74a04 admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:51.722780 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735f495e-a49c-49d6-9a68-6897bb89c872 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:51.723393 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735f495e-a49c-49d6-9a68-6897bb89c872 admin admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:51.723670 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735f495e-a49c-49d6-9a68-6897bb89c872 admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:51.723935 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735f495e-a49c-49d6-9a68-6897bb89c872 admin admin] PATH_INFO: `/v3/roles` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:51.724578 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735f495e-a49c-49d6-9a68-6897bb89c872 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:51.726353 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735f495e-a49c-49d6-9a68-6897bb89c872 admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:51.741657 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1080/2180] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 30 16:00:51 2022] GET /identity/v3/roles => generated 1420 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 16:00:51.743789 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735f495e-a49c-49d6-9a68-6897bb89c872 admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:51.754797 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbeb232-7346-4e22-853a-1eedf98740da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:51.755601 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735f495e-a49c-49d6-9a68-6897bb89c872 admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:51.755718 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbeb232-7346-4e22-853a-1eedf98740da admin admin] REQUEST_METHOD: `PUT` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:51.756027 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbeb232-7346-4e22-853a-1eedf98740da admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:51.756380 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbeb232-7346-4e22-853a-1eedf98740da admin admin] PATH_INFO: `/v3/projects/12b9adbe577c4a698ed317b9e6f942cb/users/441a522c485d4d0ca79c6fcc167bc8ba/roles/f265386950a641ac8413b53eda4ae34d` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:51.762212 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdab94d8-f6b1-4cdd-a070-6eb03b2ff8e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:51.762874 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdab94d8-f6b1-4cdd-a070-6eb03b2ff8e1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:51.763151 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdab94d8-f6b1-4cdd-a070-6eb03b2ff8e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:51.763476 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdab94d8-f6b1-4cdd-a070-6eb03b2ff8e1 admin admin] PATH_INFO: `/v3/projects/81de2fd166dc4a02a439dbffd71d1bcd/users/ce10e46249a14592a1d27b3413a4b9fc/roles/f265386950a641ac8413b53eda4ae34d` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:51.765875 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdab94d8-f6b1-4cdd-a070-6eb03b2ff8e1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=81de2fd166dc4a02a439dbffd71d1bcd, user_id=ce10e46249a14592a1d27b3413a4b9fc, role_id=f265386950a641ac8413b53eda4ae34d)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:51.770240 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdab94d8-f6b1-4cdd-a070-6eb03b2ff8e1 admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:51.779593 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbeb232-7346-4e22-853a-1eedf98740da admin admin] RBAC: Authorizing `identity:create_grant(project_id=12b9adbe577c4a698ed317b9e6f942cb, user_id=441a522c485d4d0ca79c6fcc167bc8ba, role_id=f265386950a641ac8413b53eda4ae34d)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:51.782856 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbeb232-7346-4e22-853a-1eedf98740da admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:51.784714 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1081/2181] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 30 16:00:51 2022] PUT /identity/v3/projects/81de2fd166dc4a02a439dbffd71d1bcd/users/ce10e46249a14592a1d27b3413a4b9fc/roles/f265386950a641ac8413b53eda4ae34d => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:00:51.799275 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534acb2d-40f4-455e-a6d1-003e2f551e5a None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:51.800105 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534acb2d-40f4-455e-a6d1-003e2f551e5a None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:51.800849 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534acb2d-40f4-455e-a6d1-003e2f551e5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:51.802564 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1101/2182] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 30 16:00:51 2022] PUT /identity/v3/projects/12b9adbe577c4a698ed317b9e6f942cb/users/441a522c485d4d0ca79c6fcc167bc8ba/roles/f265386950a641ac8413b53eda4ae34d => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:00:51.806366 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbeb232-7346-4e22-853a-1eedf98740da admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:51.840207 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-534acb2d-40f4-455e-a6d1-003e2f551e5a None None] MFA Rules not processed for user `ce10e46249a14592a1d27b3413a4b9fc`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:00:51.841663 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-534acb2d-40f4-455e-a6d1-003e2f551e5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:00:51.847704 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:00:51.897636 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbeb232-7346-4e22-853a-1eedf98740da admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:51.902607 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a464cf-0836-4b2f-a727-97b17d6de954 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:51.903223 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a464cf-0836-4b2f-a727-97b17d6de954 admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:51.904239 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a464cf-0836-4b2f-a727-97b17d6de954 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:51.904517 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a464cf-0836-4b2f-a727-97b17d6de954 admin admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:51.905510 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a464cf-0836-4b2f-a727-97b17d6de954 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:51.907152 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a464cf-0836-4b2f-a727-97b17d6de954 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:51.923583 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1102/2183] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 30 16:00:51 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:00:51.930398 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a464cf-0836-4b2f-a727-97b17d6de954 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:51.941349 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a464cf-0836-4b2f-a727-97b17d6de954 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:51.945977 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1082/2184] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 16:00:51 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:00:51.946438 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392bb39a-02d8-470c-9e82-74b38cd8d988 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:51.947079 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392bb39a-02d8-470c-9e82-74b38cd8d988 admin admin] REQUEST_METHOD: `PUT` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:51.947404 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392bb39a-02d8-470c-9e82-74b38cd8d988 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:51.947840 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392bb39a-02d8-470c-9e82-74b38cd8d988 admin admin] PATH_INFO: `/v3/projects/12b9adbe577c4a698ed317b9e6f942cb/users/441a522c485d4d0ca79c6fcc167bc8ba/roles/f265386950a641ac8413b53eda4ae34d` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:51.949846 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392bb39a-02d8-470c-9e82-74b38cd8d988 admin admin] RBAC: Authorizing `identity:create_grant(project_id=12b9adbe577c4a698ed317b9e6f942cb, user_id=441a522c485d4d0ca79c6fcc167bc8ba, role_id=f265386950a641ac8413b53eda4ae34d)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:51.952500 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392bb39a-02d8-470c-9e82-74b38cd8d988 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:51.963581 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1103/2185] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 30 16:00:51 2022] PUT /identity/v3/projects/12b9adbe577c4a698ed317b9e6f942cb/users/441a522c485d4d0ca79c6fcc167bc8ba/roles/f265386950a641ac8413b53eda4ae34d => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:00:51.974885 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872db1f4-9f62-48ca-970f-39159cedd01b None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:51.975174 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872db1f4-9f62-48ca-970f-39159cedd01b None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:51.975446 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872db1f4-9f62-48ca-970f-39159cedd01b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:51.978883 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392bb39a-02d8-470c-9e82-74b38cd8d988 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:52.009504 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-872db1f4-9f62-48ca-970f-39159cedd01b None None] MFA Rules not processed for user `441a522c485d4d0ca79c6fcc167bc8ba`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:00:52.011102 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-872db1f4-9f62-48ca-970f-39159cedd01b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:00:52.043445 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392bb39a-02d8-470c-9e82-74b38cd8d988 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:52.047747 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceae0c37-4e27-4514-ab69-4a6da3a88778 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0100a04821804993866ea20f159663f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:52.048361 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceae0c37-4e27-4514-ab69-4a6da3a88778 service cinder] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:52.048635 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceae0c37-4e27-4514-ab69-4a6da3a88778 service cinder] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:52.048901 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceae0c37-4e27-4514-ab69-4a6da3a88778 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:52.098379 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1083/2186] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 16:00:51 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:00:52.119463 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceae0c37-4e27-4514-ab69-4a6da3a88778 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:52.120919 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceae0c37-4e27-4514-ab69-4a6da3a88778 service cinder] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:52.130867 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1104/2187] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 16:00:51 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 16:00:52.174444 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872db1f4-9f62-48ca-970f-39159cedd01b None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:52.176343 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-872db1f4-9f62-48ca-970f-39159cedd01b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:00:52.193256 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872db1f4-9f62-48ca-970f-39159cedd01b None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:52.196825 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590773df-096a-4534-b8c6-6c518c4f679e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0100a04821804993866ea20f159663f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:52.197842 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590773df-096a-4534-b8c6-6c518c4f679e service cinder] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:52.197842 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590773df-096a-4534-b8c6-6c518c4f679e service cinder] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:52.197976 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590773df-096a-4534-b8c6-6c518c4f679e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:52.212800 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:00:52.254680 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415683a7-6cc9-490e-b68e-18629be6ca2a None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:52.254773 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415683a7-6cc9-490e-b68e-18629be6ca2a None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:52.255026 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415683a7-6cc9-490e-b68e-18629be6ca2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:52.275219 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-415683a7-6cc9-490e-b68e-18629be6ca2a None None] MFA Rules not processed for user `07130d598131443486c2bc1c6673c45b`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:00:52.276269 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590773df-096a-4534-b8c6-6c518c4f679e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:52.276466 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-415683a7-6cc9-490e-b68e-18629be6ca2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:00:52.277861 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590773df-096a-4534-b8c6-6c518c4f679e service cinder] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:52.290560 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1084/2188] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 16:00:52 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:00:52.365521 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1105/2189] 10.222.0.20 () {58 vars in 967 bytes} [Tue Aug 30 16:00:52 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 16:00:52.381254 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590773df-096a-4534-b8c6-6c518c4f679e service cinder] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:52.468377 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590773df-096a-4534-b8c6-6c518c4f679e service cinder] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:52.474382 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe83211-0923-4ea3-ac62-3fdb91031c51 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8fdafdc39a594bb4b0aeaf2fbc84c2fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:52.475051 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe83211-0923-4ea3-ac62-3fdb91031c51 service swift] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:52.475331 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe83211-0923-4ea3-ac62-3fdb91031c51 service swift] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:52.475598 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe83211-0923-4ea3-ac62-3fdb91031c51 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:52.493660 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe83211-0923-4ea3-ac62-3fdb91031c51 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:52.496885 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe83211-0923-4ea3-ac62-3fdb91031c51 service swift] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:52.512962 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1085/2190] 10.222.0.20 () {60 vars in 1384 bytes} [Tue Aug 30 16:00:52 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 30 16:00:52.876788 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40408eff-6b60-4352-9fe9-4d0876b52fb3 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:52.877292 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40408eff-6b60-4352-9fe9-4d0876b52fb3 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:52.877400 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40408eff-6b60-4352-9fe9-4d0876b52fb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:52.891163 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:00:52.897673 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-40408eff-6b60-4352-9fe9-4d0876b52fb3 None None] MFA Rules not processed for user `07130d598131443486c2bc1c6673c45b`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:00:52.900319 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-40408eff-6b60-4352-9fe9-4d0876b52fb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:00:52.910754 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1106/2191] 10.222.0.20 () {58 vars in 1009 bytes} [Tue Aug 30 16:00:52 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:00:52.919125 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe83211-0923-4ea3-ac62-3fdb91031c51 service swift] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:52.922955 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40408eff-6b60-4352-9fe9-4d0876b52fb3 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:52.934639 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40408eff-6b60-4352-9fe9-4d0876b52fb3 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:52.939728 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcae2f8-7e7a-4fd5-aec9-9e1119760b66 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8fdafdc39a594bb4b0aeaf2fbc84c2fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:52.940332 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcae2f8-7e7a-4fd5-aec9-9e1119760b66 service swift] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:52.940607 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcae2f8-7e7a-4fd5-aec9-9e1119760b66 service swift] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:52.940874 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcae2f8-7e7a-4fd5-aec9-9e1119760b66 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:52.948877 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcae2f8-7e7a-4fd5-aec9-9e1119760b66 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:52.951005 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcae2f8-7e7a-4fd5-aec9-9e1119760b66 service swift] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:52.959279 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1107/2192] 10.222.0.20 () {60 vars in 1384 bytes} [Tue Aug 30 16:00:52 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 30 16:00:52.981874 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe83211-0923-4ea3-ac62-3fdb91031c51 service swift] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:52.987287 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a09b2a-0a31-4d6d-9eb8-fa52d71a68c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:52.988272 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a09b2a-0a31-4d6d-9eb8-fa52d71a68c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:52.988549 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a09b2a-0a31-4d6d-9eb8-fa52d71a68c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:52.988817 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a09b2a-0a31-4d6d-9eb8-fa52d71a68c3 admin admin] PATH_INFO: `/v3/users/7a12b37a283740a090a45b5d76247cd1` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:52.989989 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a09b2a-0a31-4d6d-9eb8-fa52d71a68c3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7a12b37a283740a090a45b5d76247cd1)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:52.991910 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a09b2a-0a31-4d6d-9eb8-fa52d71a68c3 admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:53.043557 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:00:53.045629 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-56a09b2a-0a31-4d6d-9eb8-fa52d71a68c3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7a12b37a283740a090a45b5d76247cd1'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:53.054765 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-56a09b2a-0a31-4d6d-9eb8-fa52d71a68c3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7a12b37a283740a090a45b5d76247cd1'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:53.070844 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-56a09b2a-0a31-4d6d-9eb8-fa52d71a68c3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7a12b37a283740a090a45b5d76247cd1'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:53.077987 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-56a09b2a-0a31-4d6d-9eb8-fa52d71a68c3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7a12b37a283740a090a45b5d76247cd1'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:53.078962 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-56a09b2a-0a31-4d6d-9eb8-fa52d71a68c3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7a12b37a283740a090a45b5d76247cd1'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:53.085575 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1086/2193] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Aug 30 16:00:52 2022] DELETE /identity/v3/users/7a12b37a283740a090a45b5d76247cd1 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:00:53.090127 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2834555b-e520-4fa6-b4ae-a0b5422fbfea None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:53.090514 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2834555b-e520-4fa6-b4ae-a0b5422fbfea None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:53.090820 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2834555b-e520-4fa6-b4ae-a0b5422fbfea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:53.110419 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-2834555b-e520-4fa6-b4ae-a0b5422fbfea None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:00:53.111611 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-2834555b-e520-4fa6-b4ae-a0b5422fbfea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:00:53.175340 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1108/2194] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 16:00:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 16:00:53.185576 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a09b2a-0a31-4d6d-9eb8-fa52d71a68c3 admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:53.187183 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-56a09b2a-0a31-4d6d-9eb8-fa52d71a68c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:00:53.250565 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a09b2a-0a31-4d6d-9eb8-fa52d71a68c3 admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:53.254550 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8034a5f-c5fc-4814-b4d7-4e71ae073337 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f4157105f3488f807b7475008ab722', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:53.255158 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8034a5f-c5fc-4814-b4d7-4e71ae073337 service neutron] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:53.255431 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8034a5f-c5fc-4814-b4d7-4e71ae073337 service neutron] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:53.255695 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8034a5f-c5fc-4814-b4d7-4e71ae073337 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:53.265336 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8034a5f-c5fc-4814-b4d7-4e71ae073337 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:53.266981 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8034a5f-c5fc-4814-b4d7-4e71ae073337 service neutron] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:53.277060 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1087/2195] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 16:00:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 16:00:53.307577 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cb615d-8374-4ab2-98a5-e4882966817a None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:53.307888 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cb615d-8374-4ab2-98a5-e4882966817a None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:53.308162 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cb615d-8374-4ab2-98a5-e4882966817a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:53.328938 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-13cb615d-8374-4ab2-98a5-e4882966817a None None] MFA Rules not processed for user `07130d598131443486c2bc1c6673c45b`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:00:53.330355 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-13cb615d-8374-4ab2-98a5-e4882966817a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:00:53.409660 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1109/2196] 10.222.0.20 () {58 vars in 967 bytes} [Tue Aug 30 16:00:53 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:00:53.425806 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8034a5f-c5fc-4814-b4d7-4e71ae073337 service neutron] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:53.427374 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-e8034a5f-c5fc-4814-b4d7-4e71ae073337 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:00:53.514406 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cb615d-8374-4ab2-98a5-e4882966817a None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:53.516241 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-13cb615d-8374-4ab2-98a5-e4882966817a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:00:53.516429 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8034a5f-c5fc-4814-b4d7-4e71ae073337 service neutron] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:53.521808 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f452631-d39d-41ca-8bc0-0b38852e60b2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8fdafdc39a594bb4b0aeaf2fbc84c2fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:53.522421 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f452631-d39d-41ca-8bc0-0b38852e60b2 service swift] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:53.522694 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f452631-d39d-41ca-8bc0-0b38852e60b2 service swift] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:53.522959 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f452631-d39d-41ca-8bc0-0b38852e60b2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:53.531380 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cb615d-8374-4ab2-98a5-e4882966817a None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:53.535907 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f452631-d39d-41ca-8bc0-0b38852e60b2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:53.536462 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae01cca-b7f0-40a7-9c55-57f49adc8bf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:53.537099 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae01cca-b7f0-40a7-9c55-57f49adc8bf5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:53.537475 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae01cca-b7f0-40a7-9c55-57f49adc8bf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:53.537608 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f452631-d39d-41ca-8bc0-0b38852e60b2 service swift] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:53.537739 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae01cca-b7f0-40a7-9c55-57f49adc8bf5 admin admin] PATH_INFO: `/v3/users/b2754da700034d59a225c68f365693ce` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:53.538888 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae01cca-b7f0-40a7-9c55-57f49adc8bf5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b2754da700034d59a225c68f365693ce)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:53.540789 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae01cca-b7f0-40a7-9c55-57f49adc8bf5 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:53.546808 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1088/2197] 10.222.0.20 () {60 vars in 1384 bytes} [Tue Aug 30 16:00:53 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 30 16:00:53.581607 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f452631-d39d-41ca-8bc0-0b38852e60b2 service swift] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:53.589439 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-1f452631-d39d-41ca-8bc0-0b38852e60b2 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:00:53.604593 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f452631-d39d-41ca-8bc0-0b38852e60b2 service swift] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:53.610448 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b865f880-79ee-486a-9506-31b23b33dfed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:53.611308 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b865f880-79ee-486a-9506-31b23b33dfed admin admin] REQUEST_METHOD: `DELETE` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:53.611590 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b865f880-79ee-486a-9506-31b23b33dfed admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:53.611920 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b865f880-79ee-486a-9506-31b23b33dfed admin admin] PATH_INFO: `/v3/projects/0b57b04b591c413d8e9786b657833ac4` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:53.613948 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b865f880-79ee-486a-9506-31b23b33dfed admin admin] RBAC: Authorizing `identity:delete_project(project_id=0b57b04b591c413d8e9786b657833ac4)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:53.616208 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b865f880-79ee-486a-9506-31b23b33dfed admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:53.622350 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-2ae01cca-b7f0-40a7-9c55-57f49adc8bf5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b2754da700034d59a225c68f365693ce'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:53.665615 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-2ae01cca-b7f0-40a7-9c55-57f49adc8bf5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b2754da700034d59a225c68f365693ce'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:53.691076 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-2ae01cca-b7f0-40a7-9c55-57f49adc8bf5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b2754da700034d59a225c68f365693ce'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:53.691564 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-2ae01cca-b7f0-40a7-9c55-57f49adc8bf5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b2754da700034d59a225c68f365693ce'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:53.703311 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-b865f880-79ee-486a-9506-31b23b33dfed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0b57b04b591c413d8e9786b657833ac4', 'request_id': 'req-b865f880-79ee-486a-9506-31b23b33dfed'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:53.705099 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=92,keystone:INSERT=1,keystone:DELETE=7 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:00:53.705846 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-2ae01cca-b7f0-40a7-9c55-57f49adc8bf5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b2754da700034d59a225c68f365693ce'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:53.714625 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1110/2198] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Aug 30 16:00:53 2022] DELETE /identity/v3/users/b2754da700034d59a225c68f365693ce => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 16:00:53.719704 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3ab27c-f554-4efa-94bd-ba01591e0014 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:53.719704 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3ab27c-f554-4efa-94bd-ba01591e0014 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:53.720029 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3ab27c-f554-4efa-94bd-ba01591e0014 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:53.721661 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-b865f880-79ee-486a-9506-31b23b33dfed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0b57b04b591c413d8e9786b657833ac4', 'request_id': 'req-b865f880-79ee-486a-9506-31b23b33dfed'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:53.730145 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-b865f880-79ee-486a-9506-31b23b33dfed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0b57b04b591c413d8e9786b657833ac4', 'request_id': 'req-b865f880-79ee-486a-9506-31b23b33dfed'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:53.735863 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:00:53.747248 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-b865f880-79ee-486a-9506-31b23b33dfed admin admin] The token cache is being invalidate because project 0b57b04b591c413d8e9786b657833ac4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81512) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 16:00:53.747606 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-b865f880-79ee-486a-9506-31b23b33dfed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:53.752046 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-cc3ab27c-f554-4efa-94bd-ba01591e0014 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:00:53.754172 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-cc3ab27c-f554-4efa-94bd-ba01591e0014 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:00:53.767481 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-b865f880-79ee-486a-9506-31b23b33dfed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:53.772105 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1089/2199] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Aug 30 16:00:53 2022] DELETE /identity/v3/projects/0b57b04b591c413d8e9786b657833ac4 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:00:53.776946 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b865f880-79ee-486a-9506-31b23b33dfed admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:53.779525 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-b865f880-79ee-486a-9506-31b23b33dfed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:00:53.879748 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1111/2200] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 16:00:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:00:53.881784 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b865f880-79ee-486a-9506-31b23b33dfed admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:53.887014 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f16243a-6cf6-4ed0-a566-9abb6d70c799 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:53.887906 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f16243a-6cf6-4ed0-a566-9abb6d70c799 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:53.888182 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f16243a-6cf6-4ed0-a566-9abb6d70c799 admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:53.888452 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f16243a-6cf6-4ed0-a566-9abb6d70c799 admin admin] PATH_INFO: `/v3/users/ce10e46249a14592a1d27b3413a4b9fc` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:53.890386 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f16243a-6cf6-4ed0-a566-9abb6d70c799 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ce10e46249a14592a1d27b3413a4b9fc)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:53.891817 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3ab27c-f554-4efa-94bd-ba01591e0014 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:53.892671 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f16243a-6cf6-4ed0-a566-9abb6d70c799 admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:53.893645 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-cc3ab27c-f554-4efa-94bd-ba01591e0014 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:00:53.966370 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-4f16243a-6cf6-4ed0-a566-9abb6d70c799 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ce10e46249a14592a1d27b3413a4b9fc'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:53.991231 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-4f16243a-6cf6-4ed0-a566-9abb6d70c799 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ce10e46249a14592a1d27b3413a4b9fc'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:54.007459 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-4f16243a-6cf6-4ed0-a566-9abb6d70c799 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ce10e46249a14592a1d27b3413a4b9fc'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:54.016740 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-4f16243a-6cf6-4ed0-a566-9abb6d70c799 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ce10e46249a14592a1d27b3413a4b9fc'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:54.017546 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-4f16243a-6cf6-4ed0-a566-9abb6d70c799 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ce10e46249a14592a1d27b3413a4b9fc'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:54.018471 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3ab27c-f554-4efa-94bd-ba01591e0014 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:54.025070 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1090/2201] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Aug 30 16:00:53 2022] DELETE /identity/v3/users/ce10e46249a14592a1d27b3413a4b9fc => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:00:54.035533 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932a0fbb-cb65-4a80-a6ea-efeeba8e26e1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f4157105f3488f807b7475008ab722', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:54.035533 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932a0fbb-cb65-4a80-a6ea-efeeba8e26e1 service neutron] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:54.035533 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932a0fbb-cb65-4a80-a6ea-efeeba8e26e1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:54.035533 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932a0fbb-cb65-4a80-a6ea-efeeba8e26e1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:54.059465 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-932a0fbb-cb65-4a80-a6ea-efeeba8e26e1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:00:54.092753 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f16243a-6cf6-4ed0-a566-9abb6d70c799 admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:54.101759 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932a0fbb-cb65-4a80-a6ea-efeeba8e26e1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:54.101759 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932a0fbb-cb65-4a80-a6ea-efeeba8e26e1 service neutron] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:54.114728 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-4f16243a-6cf6-4ed0-a566-9abb6d70c799 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:00:54.126928 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1112/2202] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 16:00:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 236 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:00:54.203899 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f16243a-6cf6-4ed0-a566-9abb6d70c799 admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:54.208610 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463ee42d-3988-454b-bd6b-8282c659629c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0100a04821804993866ea20f159663f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:54.209341 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463ee42d-3988-454b-bd6b-8282c659629c service cinder] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:54.209636 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463ee42d-3988-454b-bd6b-8282c659629c service cinder] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:54.209893 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463ee42d-3988-454b-bd6b-8282c659629c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:54.213031 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-463ee42d-3988-454b-bd6b-8282c659629c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:00:54.224730 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:INSERT=2,keystone:DELETE=13,keystone:SELECT=85 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:00:54.303545 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463ee42d-3988-454b-bd6b-8282c659629c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:54.306125 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463ee42d-3988-454b-bd6b-8282c659629c service cinder] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:54.322109 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1091/2203] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 16:00:54 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 228 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 16:00:54.489072 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932a0fbb-cb65-4a80-a6ea-efeeba8e26e1 service neutron] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:54.489691 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-932a0fbb-cb65-4a80-a6ea-efeeba8e26e1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:00:54.523287 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932a0fbb-cb65-4a80-a6ea-efeeba8e26e1 service neutron] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:54.583754 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463ee42d-3988-454b-bd6b-8282c659629c service cinder] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:54.585331 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-463ee42d-3988-454b-bd6b-8282c659629c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:00:54.628490 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f973c8-3ce5-4e55-acbd-88c15e1fd9d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:54.630574 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f973c8-3ce5-4e55-acbd-88c15e1fd9d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:54.630858 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f973c8-3ce5-4e55-acbd-88c15e1fd9d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:54.631130 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f973c8-3ce5-4e55-acbd-88c15e1fd9d8 admin admin] PATH_INFO: `/v3/projects/81de2fd166dc4a02a439dbffd71d1bcd` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:54.634665 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f973c8-3ce5-4e55-acbd-88c15e1fd9d8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=81de2fd166dc4a02a439dbffd71d1bcd)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:54.643963 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f973c8-3ce5-4e55-acbd-88c15e1fd9d8 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:54.695038 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463ee42d-3988-454b-bd6b-8282c659629c service cinder] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:54.708457 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-74f973c8-3ce5-4e55-acbd-88c15e1fd9d8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '81de2fd166dc4a02a439dbffd71d1bcd', 'request_id': 'req-74f973c8-3ce5-4e55-acbd-88c15e1fd9d8'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:54.711039 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:00:54.725403 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-74f973c8-3ce5-4e55-acbd-88c15e1fd9d8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '81de2fd166dc4a02a439dbffd71d1bcd', 'request_id': 'req-74f973c8-3ce5-4e55-acbd-88c15e1fd9d8'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:54.733790 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-74f973c8-3ce5-4e55-acbd-88c15e1fd9d8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '81de2fd166dc4a02a439dbffd71d1bcd', 'request_id': 'req-74f973c8-3ce5-4e55-acbd-88c15e1fd9d8'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:54.750871 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-74f973c8-3ce5-4e55-acbd-88c15e1fd9d8 admin admin] The token cache is being invalidate because project 81de2fd166dc4a02a439dbffd71d1bcd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81511) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 16:00:54.751226 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-74f973c8-3ce5-4e55-acbd-88c15e1fd9d8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:54.752060 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-74f973c8-3ce5-4e55-acbd-88c15e1fd9d8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:54.754201 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1113/2204] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Aug 30 16:00:54 2022] DELETE /identity/v3/projects/81de2fd166dc4a02a439dbffd71d1bcd => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:00:54.758951 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b31f2d8-6cd3-4bb5-b1e4-956718c07c4a None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:54.759298 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b31f2d8-6cd3-4bb5-b1e4-956718c07c4a None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:54.759521 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b31f2d8-6cd3-4bb5-b1e4-956718c07c4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:54.777780 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79dce478-d616-4159-b2b5-1b57d3664739 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:54.778548 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dce478-d616-4159-b2b5-1b57d3664739 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:54.778850 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dce478-d616-4159-b2b5-1b57d3664739 admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:54.779154 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dce478-d616-4159-b2b5-1b57d3664739 admin admin] PATH_INFO: `/v3/projects/3d00d1a960d04391b44d0ae5084df605` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:54.780277 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79dce478-d616-4159-b2b5-1b57d3664739 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3d00d1a960d04391b44d0ae5084df605)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:54.781498 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-9b31f2d8-6cd3-4bb5-b1e4-956718c07c4a None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:00:54.782304 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79dce478-d616-4159-b2b5-1b57d3664739 admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:54.782880 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-9b31f2d8-6cd3-4bb5-b1e4-956718c07c4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:00:54.792192 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1114/2205] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 16:00:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 16:00:54.801293 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba073381-1096-451b-b51c-a6fe49edae37 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:54.801557 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba073381-1096-451b-b51c-a6fe49edae37 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:54.801825 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba073381-1096-451b-b51c-a6fe49edae37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:54.823092 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-ba073381-1096-451b-b51c-a6fe49edae37 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:00:54.824424 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-ba073381-1096-451b-b51c-a6fe49edae37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:00:54.836067 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1115/2206] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 16:00:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 16:00:54.840559 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba073381-1096-451b-b51c-a6fe49edae37 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:54.851842 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-79dce478-d616-4159-b2b5-1b57d3664739 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3d00d1a960d04391b44d0ae5084df605', 'request_id': 'req-79dce478-d616-4159-b2b5-1b57d3664739'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:54.859697 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba073381-1096-451b-b51c-a6fe49edae37 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:54.868733 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-79dce478-d616-4159-b2b5-1b57d3664739 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3d00d1a960d04391b44d0ae5084df605', 'request_id': 'req-79dce478-d616-4159-b2b5-1b57d3664739'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:54.880723 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-79dce478-d616-4159-b2b5-1b57d3664739 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3d00d1a960d04391b44d0ae5084df605', 'request_id': 'req-79dce478-d616-4159-b2b5-1b57d3664739'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:54.882509 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:00:54.897199 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-79dce478-d616-4159-b2b5-1b57d3664739 admin admin] The token cache is being invalidate because project 3d00d1a960d04391b44d0ae5084df605 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81512) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 16:00:54.897812 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-79dce478-d616-4159-b2b5-1b57d3664739 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:54.898660 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-79dce478-d616-4159-b2b5-1b57d3664739 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:54.901435 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1092/2207] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Aug 30 16:00:54 2022] DELETE /identity/v3/projects/3d00d1a960d04391b44d0ae5084df605 => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:00:54.905594 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79dce478-d616-4159-b2b5-1b57d3664739 admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:54.908829 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-79dce478-d616-4159-b2b5-1b57d3664739 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:00:54.951678 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cfdee9-60fa-4fa6-83bd-bbabbe406b22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:54.951678 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cfdee9-60fa-4fa6-83bd-bbabbe406b22 admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:54.951678 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cfdee9-60fa-4fa6-83bd-bbabbe406b22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:54.951678 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cfdee9-60fa-4fa6-83bd-bbabbe406b22 admin admin] PATH_INFO: `/v3/domains` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:54.952472 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47cfdee9-60fa-4fa6-83bd-bbabbe406b22 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:00:54.952472 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47cfdee9-60fa-4fa6-83bd-bbabbe406b22 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:54.952472 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47cfdee9-60fa-4fa6-83bd-bbabbe406b22 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:54.971545 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1116/2208] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Aug 30 16:00:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:00:54.972440 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cfdee9-60fa-4fa6-83bd-bbabbe406b22 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:54.974240 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-47cfdee9-60fa-4fa6-83bd-bbabbe406b22 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:00:54.989160 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cfdee9-60fa-4fa6-83bd-bbabbe406b22 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:54.996035 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9477684d-b55a-42ca-a9aa-d2abcff772c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:54.996854 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9477684d-b55a-42ca-a9aa-d2abcff772c7 admin admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:54.997334 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9477684d-b55a-42ca-a9aa-d2abcff772c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:54.997964 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9477684d-b55a-42ca-a9aa-d2abcff772c7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:54.999246 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9477684d-b55a-42ca-a9aa-d2abcff772c7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:55.002476 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9477684d-b55a-42ca-a9aa-d2abcff772c7 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:55.022759 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79dce478-d616-4159-b2b5-1b57d3664739 admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:55.029419 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9140ee-ca47-410b-aa9c-7df097c7f1a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:55.030582 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9140ee-ca47-410b-aa9c-7df097c7f1a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:55.030907 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9140ee-ca47-410b-aa9c-7df097c7f1a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:55.031556 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9140ee-ca47-410b-aa9c-7df097c7f1a8 admin admin] PATH_INFO: `/v3/users/82bcfe79999e4cd4a1d27c60c27798a8` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:55.032717 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9140ee-ca47-410b-aa9c-7df097c7f1a8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=82bcfe79999e4cd4a1d27c60c27798a8)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:55.034950 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9140ee-ca47-410b-aa9c-7df097c7f1a8 admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:55.042494 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1117/2209] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Aug 30 16:00:54 2022] POST /identity/v3/projects => generated 359 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:00:55.047737 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9477684d-b55a-42ca-a9aa-d2abcff772c7 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:55.105355 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-2a9140ee-ca47-410b-aa9c-7df097c7f1a8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '82bcfe79999e4cd4a1d27c60c27798a8'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:55.142004 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-2a9140ee-ca47-410b-aa9c-7df097c7f1a8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '82bcfe79999e4cd4a1d27c60c27798a8'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:55.146340 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9477684d-b55a-42ca-a9aa-d2abcff772c7 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:55.151434 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37a0435-2851-493b-a537-1532d059f419 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:55.153866 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37a0435-2851-493b-a537-1532d059f419 admin admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:55.154154 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37a0435-2851-493b-a537-1532d059f419 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:55.154456 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37a0435-2851-493b-a537-1532d059f419 admin admin] PATH_INFO: `/v3/users` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:55.155211 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37a0435-2851-493b-a537-1532d059f419 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:55.157610 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37a0435-2851-493b-a537-1532d059f419 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:55.170295 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-2a9140ee-ca47-410b-aa9c-7df097c7f1a8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '82bcfe79999e4cd4a1d27c60c27798a8'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:55.183697 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-2a9140ee-ca47-410b-aa9c-7df097c7f1a8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '82bcfe79999e4cd4a1d27c60c27798a8'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:55.184537 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-2a9140ee-ca47-410b-aa9c-7df097c7f1a8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '82bcfe79999e4cd4a1d27c60c27798a8'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:55.185556 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1118/2210] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Aug 30 16:00:55 2022] POST /identity/v3/users => generated 280 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:00:55.186357 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37a0435-2851-493b-a537-1532d059f419 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:55.188047 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-e37a0435-2851-493b-a537-1532d059f419 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:00:55.199839 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1093/2211] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Aug 30 16:00:54 2022] DELETE /identity/v3/users/82bcfe79999e4cd4a1d27c60c27798a8 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:00:55.204611 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9140ee-ca47-410b-aa9c-7df097c7f1a8 admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:55.206302 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-2a9140ee-ca47-410b-aa9c-7df097c7f1a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:00:55.225897 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:INSERT=5,keystone:DELETE=3,keystone:SELECT=92 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:00:55.291277 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9140ee-ca47-410b-aa9c-7df097c7f1a8 admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:55.293394 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37a0435-2851-493b-a537-1532d059f419 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:55.298634 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3aa550-c175-4ca4-bd60-231d756fddd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:55.299365 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3aa550-c175-4ca4-bd60-231d756fddd7 admin admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:55.299762 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3aa550-c175-4ca4-bd60-231d756fddd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:55.300064 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3aa550-c175-4ca4-bd60-231d756fddd7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:55.300908 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3aa550-c175-4ca4-bd60-231d756fddd7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:55.302802 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3aa550-c175-4ca4-bd60-231d756fddd7 admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:55.317265 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1094/2212] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 30 16:00:55 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:00:55.321431 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3aa550-c175-4ca4-bd60-231d756fddd7 admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:55.334117 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:00:55.346230 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3aa550-c175-4ca4-bd60-231d756fddd7 admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:55.359412 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce5f2c0-6693-4afe-be83-bd756ac1948a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:55.360026 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce5f2c0-6693-4afe-be83-bd756ac1948a admin admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:55.360306 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce5f2c0-6693-4afe-be83-bd756ac1948a admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:55.360575 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce5f2c0-6693-4afe-be83-bd756ac1948a admin admin] PATH_INFO: `/v3/roles` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:55.362556 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce5f2c0-6693-4afe-be83-bd756ac1948a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:55.366169 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce5f2c0-6693-4afe-be83-bd756ac1948a admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:55.389699 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c81ef9f-29b3-4a68-8f75-3a05c9d92df7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0100a04821804993866ea20f159663f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:55.390707 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c81ef9f-29b3-4a68-8f75-3a05c9d92df7 service cinder] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:55.391364 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c81ef9f-29b3-4a68-8f75-3a05c9d92df7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:55.391844 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c81ef9f-29b3-4a68-8f75-3a05c9d92df7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:55.394858 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-2c81ef9f-29b3-4a68-8f75-3a05c9d92df7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:00:55.396488 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1095/2213] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 30 16:00:55 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 16:00:55.401000 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce5f2c0-6693-4afe-be83-bd756ac1948a admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:55.416978 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce5f2c0-6693-4afe-be83-bd756ac1948a admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:55.424165 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8276f0f4-b54b-40c1-acb2-8677ef89fa60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:55.424971 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8276f0f4-b54b-40c1-acb2-8677ef89fa60 admin admin] REQUEST_METHOD: `PUT` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:55.425441 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8276f0f4-b54b-40c1-acb2-8677ef89fa60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:55.425791 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8276f0f4-b54b-40c1-acb2-8677ef89fa60 admin admin] PATH_INFO: `/v3/projects/283145e1429f499987bf7bbc45d61c54/users/41412ef89793453da4b2a59c304a8d45/roles/df8f9f844b1b439181d75a07c1a93418` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:55.458756 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8276f0f4-b54b-40c1-acb2-8677ef89fa60 admin admin] RBAC: Authorizing `identity:create_grant(project_id=283145e1429f499987bf7bbc45d61c54, user_id=41412ef89793453da4b2a59c304a8d45, role_id=df8f9f844b1b439181d75a07c1a93418)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:55.462364 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8276f0f4-b54b-40c1-acb2-8677ef89fa60 admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:55.482060 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1096/2214] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 30 16:00:55 2022] PUT /identity/v3/projects/283145e1429f499987bf7bbc45d61c54/users/41412ef89793453da4b2a59c304a8d45/roles/df8f9f844b1b439181d75a07c1a93418 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 16:00:55.493197 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7221b093-69a2-4644-824d-b9bcf51545f5 None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:55.493578 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7221b093-69a2-4644-824d-b9bcf51545f5 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:55.493914 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7221b093-69a2-4644-824d-b9bcf51545f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:55.513847 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c81ef9f-29b3-4a68-8f75-3a05c9d92df7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:55.516455 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c81ef9f-29b3-4a68-8f75-3a05c9d92df7 service cinder] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:55.531323 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-7221b093-69a2-4644-824d-b9bcf51545f5 None None] MFA Rules not processed for user `41412ef89793453da4b2a59c304a8d45`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:00:55.533158 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-7221b093-69a2-4644-824d-b9bcf51545f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:00:55.542275 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1119/2215] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 16:00:55 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:00:55.580217 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c81ef9f-29b3-4a68-8f75-3a05c9d92df7 service cinder] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:55.581781 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-2c81ef9f-29b3-4a68-8f75-3a05c9d92df7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:00:55.596425 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c81ef9f-29b3-4a68-8f75-3a05c9d92df7 service cinder] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:55.676321 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1097/2216] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 16:00:55 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:00:55.686904 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc320f9e-86b0-4d39-9877-f595bb13a46e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:55.688575 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc320f9e-86b0-4d39-9877-f595bb13a46e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:55.688996 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc320f9e-86b0-4d39-9877-f595bb13a46e admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:55.689313 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc320f9e-86b0-4d39-9877-f595bb13a46e admin admin] PATH_INFO: `/v3/projects/de0cb52f3308495f919adf22f646983f` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:55.690670 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc320f9e-86b0-4d39-9877-f595bb13a46e admin admin] RBAC: Authorizing `identity:delete_project(project_id=de0cb52f3308495f919adf22f646983f)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:55.692587 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc320f9e-86b0-4d39-9877-f595bb13a46e admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:55.723186 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7221b093-69a2-4644-824d-b9bcf51545f5 None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:55.724558 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-7221b093-69a2-4644-824d-b9bcf51545f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:00:55.735519 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:00:55.749834 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-cc320f9e-86b0-4d39-9877-f595bb13a46e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'de0cb52f3308495f919adf22f646983f', 'request_id': 'req-cc320f9e-86b0-4d39-9877-f595bb13a46e'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:55.761531 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-cc320f9e-86b0-4d39-9877-f595bb13a46e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'de0cb52f3308495f919adf22f646983f', 'request_id': 'req-cc320f9e-86b0-4d39-9877-f595bb13a46e'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:55.770476 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-cc320f9e-86b0-4d39-9877-f595bb13a46e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'de0cb52f3308495f919adf22f646983f', 'request_id': 'req-cc320f9e-86b0-4d39-9877-f595bb13a46e'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:55.783918 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-cc320f9e-86b0-4d39-9877-f595bb13a46e admin admin] The token cache is being invalidate because project de0cb52f3308495f919adf22f646983f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81511) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 16:00:55.784227 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-cc320f9e-86b0-4d39-9877-f595bb13a46e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:55.785150 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-cc320f9e-86b0-4d39-9877-f595bb13a46e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:55.786998 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1120/2217] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Aug 30 16:00:55 2022] DELETE /identity/v3/projects/de0cb52f3308495f919adf22f646983f => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:00:55.788151 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7221b093-69a2-4644-824d-b9bcf51545f5 None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:55.792070 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895668fd-1323-4c12-8bf0-3718e933a904 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:55.792348 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895668fd-1323-4c12-8bf0-3718e933a904 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:55.792616 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895668fd-1323-4c12-8bf0-3718e933a904 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:55.813209 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-895668fd-1323-4c12-8bf0-3718e933a904 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:00:55.814785 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-895668fd-1323-4c12-8bf0-3718e933a904 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:00:55.844507 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d044749-a6a3-444a-a470-9913cccb3839 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0100a04821804993866ea20f159663f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:55.845062 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d044749-a6a3-444a-a470-9913cccb3839 service cinder] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:55.845368 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d044749-a6a3-444a-a470-9913cccb3839 service cinder] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:55.845639 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d044749-a6a3-444a-a470-9913cccb3839 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:55.847380 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-4d044749-a6a3-444a-a470-9913cccb3839 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:00:55.879638 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1121/2218] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 16:00:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:00:55.887834 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d40d08-93bc-408a-a783-5815f5a6cff4 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:55.888003 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:00:55.888152 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d40d08-93bc-408a-a783-5815f5a6cff4 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:55.888696 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d40d08-93bc-408a-a783-5815f5a6cff4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:55.910031 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-f8d40d08-93bc-408a-a783-5815f5a6cff4 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:00:55.911067 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-f8d40d08-93bc-408a-a783-5815f5a6cff4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:00:55.920370 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1122/2219] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 16:00:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:00:55.924449 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d40d08-93bc-408a-a783-5815f5a6cff4 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:55.926429 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d044749-a6a3-444a-a470-9913cccb3839 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:55.928083 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d044749-a6a3-444a-a470-9913cccb3839 service cinder] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:55.934177 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d40d08-93bc-408a-a783-5815f5a6cff4 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:55.939004 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d5f310-4cc6-4c3c-8f69-949aa7839dad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:55.939618 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d5f310-4cc6-4c3c-8f69-949aa7839dad admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:55.939727 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1098/2220] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 16:00:55 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 16:00:55.939887 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d5f310-4cc6-4c3c-8f69-949aa7839dad admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:55.940159 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d5f310-4cc6-4c3c-8f69-949aa7839dad admin admin] PATH_INFO: `/v3/domains` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:55.940871 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d5f310-4cc6-4c3c-8f69-949aa7839dad admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:00:55.941168 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d5f310-4cc6-4c3c-8f69-949aa7839dad admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:55.942880 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d5f310-4cc6-4c3c-8f69-949aa7839dad admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:55.955137 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1123/2221] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Aug 30 16:00:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 16:00:55.962339 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d5f310-4cc6-4c3c-8f69-949aa7839dad admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:55.964014 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d044749-a6a3-444a-a470-9913cccb3839 service cinder] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:55.966156 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-4d044749-a6a3-444a-a470-9913cccb3839 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:00:55.973616 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d5f310-4cc6-4c3c-8f69-949aa7839dad admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:55.979340 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd24516-7fb8-4a2f-b2c5-5c996efda8c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:55.979913 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd24516-7fb8-4a2f-b2c5-5c996efda8c0 admin admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:55.980199 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd24516-7fb8-4a2f-b2c5-5c996efda8c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:55.980465 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd24516-7fb8-4a2f-b2c5-5c996efda8c0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:55.981116 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd24516-7fb8-4a2f-b2c5-5c996efda8c0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:55.984277 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd24516-7fb8-4a2f-b2c5-5c996efda8c0 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:55.989638 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d044749-a6a3-444a-a470-9913cccb3839 service cinder] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:55.995507 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedf9439-9aca-4f1b-a021-ac469bb8faee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:55.996654 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedf9439-9aca-4f1b-a021-ac469bb8faee admin admin] REQUEST_METHOD: `DELETE` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:55.997040 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedf9439-9aca-4f1b-a021-ac469bb8faee admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:55.997450 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedf9439-9aca-4f1b-a021-ac469bb8faee admin admin] PATH_INFO: `/v3/users/41412ef89793453da4b2a59c304a8d45` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:55.999533 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dedf9439-9aca-4f1b-a021-ac469bb8faee admin admin] RBAC: Authorizing `identity:delete_user(user_id=41412ef89793453da4b2a59c304a8d45)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:56.001760 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dedf9439-9aca-4f1b-a021-ac469bb8faee admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:56.016059 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1124/2222] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Aug 30 16:00:55 2022] POST /identity/v3/projects => generated 385 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 16:00:56.020262 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd24516-7fb8-4a2f-b2c5-5c996efda8c0 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:56.054430 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-dedf9439-9aca-4f1b-a021-ac469bb8faee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '41412ef89793453da4b2a59c304a8d45'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:56.062328 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-dedf9439-9aca-4f1b-a021-ac469bb8faee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '41412ef89793453da4b2a59c304a8d45'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:56.076429 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-dedf9439-9aca-4f1b-a021-ac469bb8faee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '41412ef89793453da4b2a59c304a8d45'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:56.085716 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-dedf9439-9aca-4f1b-a021-ac469bb8faee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '41412ef89793453da4b2a59c304a8d45'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:56.086481 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-dedf9439-9aca-4f1b-a021-ac469bb8faee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '41412ef89793453da4b2a59c304a8d45'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:56.088231 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd24516-7fb8-4a2f-b2c5-5c996efda8c0 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:56.092569 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1099/2223] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Aug 30 16:00:55 2022] DELETE /identity/v3/users/41412ef89793453da4b2a59c304a8d45 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:00:56.092902 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c95b9d-3ad7-449e-acd0-10da6a26025e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:56.093540 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c95b9d-3ad7-449e-acd0-10da6a26025e admin admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:56.093815 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c95b9d-3ad7-449e-acd0-10da6a26025e admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:56.094237 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c95b9d-3ad7-449e-acd0-10da6a26025e admin admin] PATH_INFO: `/v3/users` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:56.094869 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c95b9d-3ad7-449e-acd0-10da6a26025e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:56.096806 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c95b9d-3ad7-449e-acd0-10da6a26025e admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:56.097069 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4103ec-fbfa-4261-adad-9719110ff08a None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:56.097474 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4103ec-fbfa-4261-adad-9719110ff08a None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:56.097807 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4103ec-fbfa-4261-adad-9719110ff08a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:56.119589 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-8c4103ec-fbfa-4261-adad-9719110ff08a None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:00:56.120857 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-8c4103ec-fbfa-4261-adad-9719110ff08a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:00:56.125360 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1125/2224] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Aug 30 16:00:56 2022] POST /identity/v3/users => generated 293 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 16:00:56.129154 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c95b9d-3ad7-449e-acd0-10da6a26025e admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:56.131818 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-a9c95b9d-3ad7-449e-acd0-10da6a26025e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:00:56.147620 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c95b9d-3ad7-449e-acd0-10da6a26025e admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:56.155738 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:DELETE=5,keystone:SELECT=90,keystone:INSERT=5 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:00:56.190952 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1100/2225] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 16:00:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 16:00:56.201037 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4103ec-fbfa-4261-adad-9719110ff08a None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:56.202948 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-8c4103ec-fbfa-4261-adad-9719110ff08a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:00:56.209062 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfea58e-bc4f-490c-bc65-cc153fe50279 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:56.209700 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfea58e-bc4f-490c-bc65-cc153fe50279 admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:56.209977 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfea58e-bc4f-490c-bc65-cc153fe50279 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:56.210248 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfea58e-bc4f-490c-bc65-cc153fe50279 admin admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:56.210980 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dfea58e-bc4f-490c-bc65-cc153fe50279 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:56.212684 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dfea58e-bc4f-490c-bc65-cc153fe50279 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:56.222697 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1126/2226] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 30 16:00:56 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:00:56.226593 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfea58e-bc4f-490c-bc65-cc153fe50279 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:56.235994 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfea58e-bc4f-490c-bc65-cc153fe50279 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:56.241396 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1061e40b-9769-4fdb-b965-99975512cdf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:56.241978 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1061e40b-9769-4fdb-b965-99975512cdf8 admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:56.242256 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1061e40b-9769-4fdb-b965-99975512cdf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:56.242527 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1061e40b-9769-4fdb-b965-99975512cdf8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:56.243159 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1061e40b-9769-4fdb-b965-99975512cdf8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:56.244729 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1061e40b-9769-4fdb-b965-99975512cdf8 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:56.250328 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:00:56.255289 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1127/2227] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 30 16:00:56 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 16:00:56.260054 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1061e40b-9769-4fdb-b965-99975512cdf8 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:56.263139 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4103ec-fbfa-4261-adad-9719110ff08a None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:56.269475 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abda4b9-a5fe-446e-92e7-247fb1a3a930 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f4157105f3488f807b7475008ab722', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:56.270405 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abda4b9-a5fe-446e-92e7-247fb1a3a930 service neutron] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:56.270671 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abda4b9-a5fe-446e-92e7-247fb1a3a930 service neutron] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:56.270933 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abda4b9-a5fe-446e-92e7-247fb1a3a930 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:56.276402 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1061e40b-9769-4fdb-b965-99975512cdf8 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:56.283149 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d012c9cb-9502-4dd0-a0c8-6001b2ce8b5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:56.283816 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d012c9cb-9502-4dd0-a0c8-6001b2ce8b5b admin admin] REQUEST_METHOD: `PUT` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:56.284094 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d012c9cb-9502-4dd0-a0c8-6001b2ce8b5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:56.284441 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d012c9cb-9502-4dd0-a0c8-6001b2ce8b5b admin admin] PATH_INFO: `/v3/projects/515055be6c664727847ab4eae19dd04f/users/7ba3d0615e5a4c8cb2367eba59cbaee9/roles/df8f9f844b1b439181d75a07c1a93418` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:56.284933 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abda4b9-a5fe-446e-92e7-247fb1a3a930 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:56.286467 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abda4b9-a5fe-446e-92e7-247fb1a3a930 service neutron] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:56.298290 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1101/2228] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 16:00:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 16:00:56.299532 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abda4b9-a5fe-446e-92e7-247fb1a3a930 service neutron] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:56.301661 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-6abda4b9-a5fe-446e-92e7-247fb1a3a930 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:00:56.304595 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d012c9cb-9502-4dd0-a0c8-6001b2ce8b5b admin admin] RBAC: Authorizing `identity:create_grant(project_id=515055be6c664727847ab4eae19dd04f, user_id=7ba3d0615e5a4c8cb2367eba59cbaee9, role_id=df8f9f844b1b439181d75a07c1a93418)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:56.307312 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d012c9cb-9502-4dd0-a0c8-6001b2ce8b5b admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:56.315322 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abda4b9-a5fe-446e-92e7-247fb1a3a930 service neutron] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:56.321116 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cc160c-7542-4748-85cb-afc6aaabc6b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:56.321821 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cc160c-7542-4748-85cb-afc6aaabc6b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:56.322140 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cc160c-7542-4748-85cb-afc6aaabc6b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:56.322447 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cc160c-7542-4748-85cb-afc6aaabc6b2 admin admin] PATH_INFO: `/v3/users/af9f01e25ee947db9c11cc4577e980ea` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:56.324108 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cc160c-7542-4748-85cb-afc6aaabc6b2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=af9f01e25ee947db9c11cc4577e980ea)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:56.326452 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cc160c-7542-4748-85cb-afc6aaabc6b2 admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:56.328996 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1128/2229] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 30 16:00:56 2022] PUT /identity/v3/projects/515055be6c664727847ab4eae19dd04f/users/7ba3d0615e5a4c8cb2367eba59cbaee9/roles/df8f9f844b1b439181d75a07c1a93418 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 16:00:56.390903 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-d8cc160c-7542-4748-85cb-afc6aaabc6b2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'af9f01e25ee947db9c11cc4577e980ea'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:56.405006 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-d8cc160c-7542-4748-85cb-afc6aaabc6b2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'af9f01e25ee947db9c11cc4577e980ea'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:56.406601 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313f89d5-fcfe-4d6c-9961-865b2f4e8ab6 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:56.406881 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313f89d5-fcfe-4d6c-9961-865b2f4e8ab6 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:56.407173 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313f89d5-fcfe-4d6c-9961-865b2f4e8ab6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:56.429362 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-d8cc160c-7542-4748-85cb-afc6aaabc6b2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'af9f01e25ee947db9c11cc4577e980ea'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:56.438500 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-d8cc160c-7542-4748-85cb-afc6aaabc6b2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'af9f01e25ee947db9c11cc4577e980ea'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:56.439739 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-d8cc160c-7542-4748-85cb-afc6aaabc6b2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'af9f01e25ee947db9c11cc4577e980ea'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:56.446674 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1102/2230] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Aug 30 16:00:56 2022] DELETE /identity/v3/users/af9f01e25ee947db9c11cc4577e980ea => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:00:56.447591 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-313f89d5-fcfe-4d6c-9961-865b2f4e8ab6 None None] MFA Rules not processed for user `7ba3d0615e5a4c8cb2367eba59cbaee9`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:00:56.448948 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-313f89d5-fcfe-4d6c-9961-865b2f4e8ab6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:00:56.451179 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57d0619-6c51-4f31-922c-3d528b88ee50 None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:56.451971 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57d0619-6c51-4f31-922c-3d528b88ee50 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:56.452241 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57d0619-6c51-4f31-922c-3d528b88ee50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:56.479744 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-a57d0619-6c51-4f31-922c-3d528b88ee50 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:00:56.481281 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-a57d0619-6c51-4f31-922c-3d528b88ee50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:00:56.536726 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1129/2231] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 16:00:56 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 16:00:56.549440 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313f89d5-fcfe-4d6c-9961-865b2f4e8ab6 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:56.551091 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-313f89d5-fcfe-4d6c-9961-865b2f4e8ab6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:00:56.572181 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1103/2232] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 16:00:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:00:56.586461 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57d0619-6c51-4f31-922c-3d528b88ee50 None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:56.588579 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-a57d0619-6c51-4f31-922c-3d528b88ee50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:00:56.644003 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313f89d5-fcfe-4d6c-9961-865b2f4e8ab6 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:56.651845 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9667895a-2b0b-4c86-b977-0984fd12463a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0100a04821804993866ea20f159663f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:56.652940 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9667895a-2b0b-4c86-b977-0984fd12463a service cinder] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:56.653511 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9667895a-2b0b-4c86-b977-0984fd12463a service cinder] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:56.653801 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9667895a-2b0b-4c86-b977-0984fd12463a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:56.664257 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9667895a-2b0b-4c86-b977-0984fd12463a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:56.665747 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9667895a-2b0b-4c86-b977-0984fd12463a service cinder] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:56.670746 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57d0619-6c51-4f31-922c-3d528b88ee50 None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:56.675360 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fb53cb-ae68-4e13-b7fe-1faa33db9a0f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f4157105f3488f807b7475008ab722', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:56.675965 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fb53cb-ae68-4e13-b7fe-1faa33db9a0f service neutron] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:56.676242 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fb53cb-ae68-4e13-b7fe-1faa33db9a0f service neutron] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:56.676346 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1130/2233] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 16:00:56 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 16:00:56.676684 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fb53cb-ae68-4e13-b7fe-1faa33db9a0f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:56.677074 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9667895a-2b0b-4c86-b977-0984fd12463a service cinder] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:56.678468 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-9667895a-2b0b-4c86-b977-0984fd12463a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:00:56.691430 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:00:56.691585 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82fb53cb-ae68-4e13-b7fe-1faa33db9a0f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:56.693145 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82fb53cb-ae68-4e13-b7fe-1faa33db9a0f service neutron] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:56.697013 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9667895a-2b0b-4c86-b977-0984fd12463a service cinder] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:56.700648 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:00:56.705080 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcd2397-cdab-4c8d-b8e0-73cc55ada546 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:56.705745 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcd2397-cdab-4c8d-b8e0-73cc55ada546 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:56.706039 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcd2397-cdab-4c8d-b8e0-73cc55ada546 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:56.706477 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcd2397-cdab-4c8d-b8e0-73cc55ada546 admin admin] PATH_INFO: `/v3/projects/283145e1429f499987bf7bbc45d61c54` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:56.707740 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efcd2397-cdab-4c8d-b8e0-73cc55ada546 admin admin] RBAC: Authorizing `identity:delete_project(project_id=283145e1429f499987bf7bbc45d61c54)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:56.709526 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efcd2397-cdab-4c8d-b8e0-73cc55ada546 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:56.709640 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1104/2234] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 16:00:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 16:00:56.770645 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-efcd2397-cdab-4c8d-b8e0-73cc55ada546 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '283145e1429f499987bf7bbc45d61c54', 'request_id': 'req-efcd2397-cdab-4c8d-b8e0-73cc55ada546'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:56.780851 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-efcd2397-cdab-4c8d-b8e0-73cc55ada546 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '283145e1429f499987bf7bbc45d61c54', 'request_id': 'req-efcd2397-cdab-4c8d-b8e0-73cc55ada546'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:56.790699 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-efcd2397-cdab-4c8d-b8e0-73cc55ada546 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '283145e1429f499987bf7bbc45d61c54', 'request_id': 'req-efcd2397-cdab-4c8d-b8e0-73cc55ada546'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:56.804767 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-efcd2397-cdab-4c8d-b8e0-73cc55ada546 admin admin] The token cache is being invalidate because project 283145e1429f499987bf7bbc45d61c54 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81511) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 16:00:56.805120 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-efcd2397-cdab-4c8d-b8e0-73cc55ada546 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:56.806078 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-efcd2397-cdab-4c8d-b8e0-73cc55ada546 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:56.808574 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1131/2235] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Aug 30 16:00:56 2022] DELETE /identity/v3/projects/283145e1429f499987bf7bbc45d61c54 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:00:56.813998 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23aa6b9-59f6-47cb-9e80-2c282ff7a9ba None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:56.814502 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23aa6b9-59f6-47cb-9e80-2c282ff7a9ba None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:56.814889 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23aa6b9-59f6-47cb-9e80-2c282ff7a9ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:56.840333 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-c23aa6b9-59f6-47cb-9e80-2c282ff7a9ba None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:00:56.842434 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-c23aa6b9-59f6-47cb-9e80-2c282ff7a9ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:00:56.934883 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcd2397-cdab-4c8d-b8e0-73cc55ada546 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:56.938600 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-efcd2397-cdab-4c8d-b8e0-73cc55ada546 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:00:56.975660 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1105/2236] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 16:00:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:00:56.983846 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0792602-1936-4efd-b935-d8f2ce7e494e None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:56.984230 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0792602-1936-4efd-b935-d8f2ce7e494e None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:56.984482 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0792602-1936-4efd-b935-d8f2ce7e494e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:57.022350 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-f0792602-1936-4efd-b935-d8f2ce7e494e None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:00:57.024241 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-f0792602-1936-4efd-b935-d8f2ce7e494e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:00:57.040830 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1106/2237] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 16:00:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 16:00:57.042545 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0792602-1936-4efd-b935-d8f2ce7e494e None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:57.075808 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcd2397-cdab-4c8d-b8e0-73cc55ada546 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:57.080231 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756255dd-7d39-4c1a-a5a7-e18e6da342d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:57.081051 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756255dd-7d39-4c1a-a5a7-e18e6da342d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:57.081433 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756255dd-7d39-4c1a-a5a7-e18e6da342d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:57.081780 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756255dd-7d39-4c1a-a5a7-e18e6da342d6 admin admin] PATH_INFO: `/v3/projects/058ab65d5dcb4bea82fe83721c58a1c6` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:57.083587 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756255dd-7d39-4c1a-a5a7-e18e6da342d6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=058ab65d5dcb4bea82fe83721c58a1c6)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:57.086263 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756255dd-7d39-4c1a-a5a7-e18e6da342d6 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:57.089619 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0792602-1936-4efd-b935-d8f2ce7e494e None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:57.097442 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110f9018-f541-4b31-8bd5-9a8ab77561a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:57.098782 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110f9018-f541-4b31-8bd5-9a8ab77561a6 admin admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:57.099064 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110f9018-f541-4b31-8bd5-9a8ab77561a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:57.099337 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110f9018-f541-4b31-8bd5-9a8ab77561a6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:57.100111 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110f9018-f541-4b31-8bd5-9a8ab77561a6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81512) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:00:57.100426 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110f9018-f541-4b31-8bd5-9a8ab77561a6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:57.102149 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110f9018-f541-4b31-8bd5-9a8ab77561a6 admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:57.128895 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1107/2238] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Aug 30 16:00:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:00:57.135757 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110f9018-f541-4b31-8bd5-9a8ab77561a6 admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:57.162066 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110f9018-f541-4b31-8bd5-9a8ab77561a6 admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:57.178564 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-756255dd-7d39-4c1a-a5a7-e18e6da342d6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '058ab65d5dcb4bea82fe83721c58a1c6', 'request_id': 'req-756255dd-7d39-4c1a-a5a7-e18e6da342d6'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:57.200824 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-756255dd-7d39-4c1a-a5a7-e18e6da342d6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '058ab65d5dcb4bea82fe83721c58a1c6', 'request_id': 'req-756255dd-7d39-4c1a-a5a7-e18e6da342d6'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:57.208172 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-756255dd-7d39-4c1a-a5a7-e18e6da342d6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '058ab65d5dcb4bea82fe83721c58a1c6', 'request_id': 'req-756255dd-7d39-4c1a-a5a7-e18e6da342d6'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:57.227927 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-756255dd-7d39-4c1a-a5a7-e18e6da342d6 admin admin] The token cache is being invalidate because project 058ab65d5dcb4bea82fe83721c58a1c6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81511) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 16:00:57.228312 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-756255dd-7d39-4c1a-a5a7-e18e6da342d6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:57.229576 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-756255dd-7d39-4c1a-a5a7-e18e6da342d6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:57.233597 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1132/2239] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Aug 30 16:00:56 2022] DELETE /identity/v3/projects/058ab65d5dcb4bea82fe83721c58a1c6 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:00:57.275675 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43290e1-734f-4f02-961d-18850784f837 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:57.276368 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43290e1-734f-4f02-961d-18850784f837 admin admin] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:57.277362 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43290e1-734f-4f02-961d-18850784f837 admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:57.277712 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43290e1-734f-4f02-961d-18850784f837 admin admin] PATH_INFO: `/v3/projects` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:57.278476 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43290e1-734f-4f02-961d-18850784f837 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:57.281809 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43290e1-734f-4f02-961d-18850784f837 admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:57.359968 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756255dd-7d39-4c1a-a5a7-e18e6da342d6 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:57.362528 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-756255dd-7d39-4c1a-a5a7-e18e6da342d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:00:57.373207 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1108/2240] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Aug 30 16:00:57 2022] POST /identity/v3/projects => generated 373 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:00:57.387427 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756255dd-7d39-4c1a-a5a7-e18e6da342d6 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:57.421705 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:00:57.485121 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e9c2bf-b47a-4f07-8857-967a2f3ef818 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:57.485897 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e9c2bf-b47a-4f07-8857-967a2f3ef818 admin admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:57.486171 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e9c2bf-b47a-4f07-8857-967a2f3ef818 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:57.486440 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e9c2bf-b47a-4f07-8857-967a2f3ef818 admin admin] PATH_INFO: `/v3/users` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:57.487074 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e9c2bf-b47a-4f07-8857-967a2f3ef818 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:57.489562 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e9c2bf-b47a-4f07-8857-967a2f3ef818 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:57.513178 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1133/2241] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Aug 30 16:00:57 2022] POST /identity/v3/users => generated 287 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:00:57.517734 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43290e1-734f-4f02-961d-18850784f837 admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:57.528156 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43290e1-734f-4f02-961d-18850784f837 admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:57.532957 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ba243f-f307-4c48-8760-077ae52f5dde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:57.533598 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ba243f-f307-4c48-8760-077ae52f5dde admin admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:57.533882 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ba243f-f307-4c48-8760-077ae52f5dde admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:57.534153 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ba243f-f307-4c48-8760-077ae52f5dde admin admin] PATH_INFO: `/v3/roles` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:57.534799 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ba243f-f307-4c48-8760-077ae52f5dde admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:57.536440 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ba243f-f307-4c48-8760-077ae52f5dde admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:57.546507 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1109/2242] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 30 16:00:57 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 16:00:57.550318 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e9c2bf-b47a-4f07-8857-967a2f3ef818 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:57.560851 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e9c2bf-b47a-4f07-8857-967a2f3ef818 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:57.566172 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a47a89-64e9-47ab-8126-531e81b67820 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:57.566801 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a47a89-64e9-47ab-8126-531e81b67820 admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:57.567079 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a47a89-64e9-47ab-8126-531e81b67820 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:57.567435 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a47a89-64e9-47ab-8126-531e81b67820 admin admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:57.568034 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a47a89-64e9-47ab-8126-531e81b67820 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:57.569672 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a47a89-64e9-47ab-8126-531e81b67820 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:57.580805 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1134/2243] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 30 16:00:57 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:00:57.584991 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ba243f-f307-4c48-8760-077ae52f5dde admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:57.596434 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ba243f-f307-4c48-8760-077ae52f5dde admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:57.602095 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a729225f-7a69-4285-83e5-99ec74e1d0cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:57.602743 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a729225f-7a69-4285-83e5-99ec74e1d0cc admin admin] REQUEST_METHOD: `PUT` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:57.603078 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a729225f-7a69-4285-83e5-99ec74e1d0cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:57.603371 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a729225f-7a69-4285-83e5-99ec74e1d0cc admin admin] PATH_INFO: `/v3/projects/80493f357d57418cb58f8a33e82b8822/users/e9fdf5c5f263474bb41ab41bdac6f78f/roles/df8f9f844b1b439181d75a07c1a93418` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:57.618376 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a729225f-7a69-4285-83e5-99ec74e1d0cc admin admin] RBAC: Authorizing `identity:create_grant(project_id=80493f357d57418cb58f8a33e82b8822, user_id=e9fdf5c5f263474bb41ab41bdac6f78f, role_id=df8f9f844b1b439181d75a07c1a93418)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:57.621045 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a729225f-7a69-4285-83e5-99ec74e1d0cc admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:57.635937 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1110/2244] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 30 16:00:57 2022] PUT /identity/v3/projects/80493f357d57418cb58f8a33e82b8822/users/e9fdf5c5f263474bb41ab41bdac6f78f/roles/df8f9f844b1b439181d75a07c1a93418 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 16:00:57.641445 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf97721-acc6-4d6f-aae2-5076ca773319 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:57.641731 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf97721-acc6-4d6f-aae2-5076ca773319 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:57.642004 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf97721-acc6-4d6f-aae2-5076ca773319 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:57.662591 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-ddf97721-acc6-4d6f-aae2-5076ca773319 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:00:57.663840 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-ddf97721-acc6-4d6f-aae2-5076ca773319 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:00:57.728729 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1135/2245] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 16:00:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:00:57.740207 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a729225f-7a69-4285-83e5-99ec74e1d0cc admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:57.742025 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-a729225f-7a69-4285-83e5-99ec74e1d0cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:00:57.775853 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:00:57.801285 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a729225f-7a69-4285-83e5-99ec74e1d0cc admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:57.804845 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be30637-3a28-4a63-8976-7eea7a8f33fa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f4157105f3488f807b7475008ab722', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:57.805487 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be30637-3a28-4a63-8976-7eea7a8f33fa service neutron] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:57.805758 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be30637-3a28-4a63-8976-7eea7a8f33fa service neutron] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:57.806030 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be30637-3a28-4a63-8976-7eea7a8f33fa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:57.814952 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be30637-3a28-4a63-8976-7eea7a8f33fa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:57.816394 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be30637-3a28-4a63-8976-7eea7a8f33fa service neutron] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:57.825632 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1111/2246] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 16:00:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 16:00:58.529075 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf89a3e3-318a-4945-9cd5-4381ee98d555 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:58.530012 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf89a3e3-318a-4945-9cd5-4381ee98d555 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:58.530466 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf89a3e3-318a-4945-9cd5-4381ee98d555 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:58.567185 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-bf89a3e3-318a-4945-9cd5-4381ee98d555 None None] MFA Rules not processed for user `e9fdf5c5f263474bb41ab41bdac6f78f`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:00:58.569074 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-bf89a3e3-318a-4945-9cd5-4381ee98d555 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:00:58.665738 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1136/2247] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 16:00:58 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:00:58.687114 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be30637-3a28-4a63-8976-7eea7a8f33fa service neutron] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:58.688652 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-1be30637-3a28-4a63-8976-7eea7a8f33fa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:00:58.774202 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be30637-3a28-4a63-8976-7eea7a8f33fa service neutron] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:58.778148 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381bc0e3-e0b4-4645-850f-79c51575278c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0100a04821804993866ea20f159663f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:58.778821 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381bc0e3-e0b4-4645-850f-79c51575278c service cinder] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:58.779159 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381bc0e3-e0b4-4645-850f-79c51575278c service cinder] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:58.779490 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381bc0e3-e0b4-4645-850f-79c51575278c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:58.790909 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381bc0e3-e0b4-4645-850f-79c51575278c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:58.792414 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381bc0e3-e0b4-4645-850f-79c51575278c service cinder] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:58.802991 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1112/2248] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 16:00:58 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 16:00:59.523868 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf89a3e3-318a-4945-9cd5-4381ee98d555 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:59.525582 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-bf89a3e3-318a-4945-9cd5-4381ee98d555 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:00:59.540709 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf89a3e3-318a-4945-9cd5-4381ee98d555 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:59.546143 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8866f9d7-193d-4863-b4d8-69fc90a0506b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:59.546930 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8866f9d7-193d-4863-b4d8-69fc90a0506b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:59.547374 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8866f9d7-193d-4863-b4d8-69fc90a0506b admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:59.547653 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8866f9d7-193d-4863-b4d8-69fc90a0506b admin admin] PATH_INFO: `/v3/users/7ba3d0615e5a4c8cb2367eba59cbaee9` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:59.548879 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8866f9d7-193d-4863-b4d8-69fc90a0506b admin admin] RBAC: Authorizing `identity:delete_user(user_id=7ba3d0615e5a4c8cb2367eba59cbaee9)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:59.550898 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8866f9d7-193d-4863-b4d8-69fc90a0506b admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:59.572966 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=94,keystone:INSERT=3,keystone:DELETE=3 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:00:59.603221 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-8866f9d7-193d-4863-b4d8-69fc90a0506b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7ba3d0615e5a4c8cb2367eba59cbaee9'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:59.616175 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-8866f9d7-193d-4863-b4d8-69fc90a0506b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7ba3d0615e5a4c8cb2367eba59cbaee9'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:59.623873 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-8866f9d7-193d-4863-b4d8-69fc90a0506b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7ba3d0615e5a4c8cb2367eba59cbaee9'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:59.624544 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-8866f9d7-193d-4863-b4d8-69fc90a0506b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7ba3d0615e5a4c8cb2367eba59cbaee9'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:59.632687 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-8866f9d7-193d-4863-b4d8-69fc90a0506b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7ba3d0615e5a4c8cb2367eba59cbaee9'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:59.639226 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1137/2249] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Aug 30 16:00:59 2022] DELETE /identity/v3/users/7ba3d0615e5a4c8cb2367eba59cbaee9 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:00:59.643927 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36070c44-da01-49c9-928f-9872f5fb8767 None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:59.644213 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36070c44-da01-49c9-928f-9872f5fb8767 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:59.644486 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36070c44-da01-49c9-928f-9872f5fb8767 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:59.663585 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-36070c44-da01-49c9-928f-9872f5fb8767 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:00:59.664788 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-36070c44-da01-49c9-928f-9872f5fb8767 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:00:59.725825 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1113/2250] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 16:00:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:00:59.736039 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8866f9d7-193d-4863-b4d8-69fc90a0506b admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:59.737486 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-8866f9d7-193d-4863-b4d8-69fc90a0506b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:00:59.805460 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8866f9d7-193d-4863-b4d8-69fc90a0506b admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:59.810330 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277f26db-c5b3-40c1-9ea1-acbed84d9d0c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f4157105f3488f807b7475008ab722', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:59.810985 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277f26db-c5b3-40c1-9ea1-acbed84d9d0c service neutron] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:59.811264 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277f26db-c5b3-40c1-9ea1-acbed84d9d0c service neutron] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:59.811536 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277f26db-c5b3-40c1-9ea1-acbed84d9d0c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:59.822558 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277f26db-c5b3-40c1-9ea1-acbed84d9d0c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:59.824160 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277f26db-c5b3-40c1-9ea1-acbed84d9d0c service neutron] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:59.834456 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1138/2251] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 16:00:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:00:59.884207 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36070c44-da01-49c9-928f-9872f5fb8767 None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:00:59.885654 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-36070c44-da01-49c9-928f-9872f5fb8767 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:00:59.898599 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36070c44-da01-49c9-928f-9872f5fb8767 None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:00:59.903683 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0df1708-5595-40f1-904d-725ecf7cdd4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:00:59.904312 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0df1708-5595-40f1-904d-725ecf7cdd4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:00:59.904588 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0df1708-5595-40f1-904d-725ecf7cdd4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:00:59.904867 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0df1708-5595-40f1-904d-725ecf7cdd4e admin admin] PATH_INFO: `/v3/users/848e370c25d04e70b04f6754a70ffa28` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:00:59.906120 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0df1708-5595-40f1-904d-725ecf7cdd4e admin admin] RBAC: Authorizing `identity:delete_user(user_id=848e370c25d04e70b04f6754a70ffa28)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:00:59.908105 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0df1708-5595-40f1-904d-725ecf7cdd4e admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:00:59.957621 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-b0df1708-5595-40f1-904d-725ecf7cdd4e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '848e370c25d04e70b04f6754a70ffa28'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:59.967180 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-b0df1708-5595-40f1-904d-725ecf7cdd4e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '848e370c25d04e70b04f6754a70ffa28'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:59.980206 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-b0df1708-5595-40f1-904d-725ecf7cdd4e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '848e370c25d04e70b04f6754a70ffa28'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:59.982354 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:00:59.991125 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-b0df1708-5595-40f1-904d-725ecf7cdd4e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '848e370c25d04e70b04f6754a70ffa28'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:00:59.992082 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-b0df1708-5595-40f1-904d-725ecf7cdd4e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '848e370c25d04e70b04f6754a70ffa28'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:01:00.000555 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1114/2252] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Aug 30 16:00:59 2022] DELETE /identity/v3/users/848e370c25d04e70b04f6754a70ffa28 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:01:00.005436 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9fa9f2-c067-467b-9932-22196f7943b8 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:01:00.006067 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9fa9f2-c067-467b-9932-22196f7943b8 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:01:00.006067 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9fa9f2-c067-467b-9932-22196f7943b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:01:00.025494 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-5b9fa9f2-c067-467b-9932-22196f7943b8 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:01:00.026722 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-5b9fa9f2-c067-467b-9932-22196f7943b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:01:00.088677 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1139/2253] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 16:01:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:01:00.099322 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0df1708-5595-40f1-904d-725ecf7cdd4e admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:01:00.101092 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-b0df1708-5595-40f1-904d-725ecf7cdd4e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:01:00.182612 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0df1708-5595-40f1-904d-725ecf7cdd4e admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:01:00.188951 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7dd14c-e3b4-40ff-ae57-29f864f6d7e8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f4157105f3488f807b7475008ab722', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:01:00.189706 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7dd14c-e3b4-40ff-ae57-29f864f6d7e8 service neutron] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:01:00.189985 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7dd14c-e3b4-40ff-ae57-29f864f6d7e8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:01:00.190375 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7dd14c-e3b4-40ff-ae57-29f864f6d7e8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:01:00.204978 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7dd14c-e3b4-40ff-ae57-29f864f6d7e8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:01:00.207093 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7dd14c-e3b4-40ff-ae57-29f864f6d7e8 service neutron] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:01:00.220184 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1115/2254] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 16:01:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:01:00.233341 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9fa9f2-c067-467b-9932-22196f7943b8 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:01:00.242094 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-5b9fa9f2-c067-467b-9932-22196f7943b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:01:00.273890 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9fa9f2-c067-467b-9932-22196f7943b8 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:01:00.275077 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e8fe61-bba6-4471-a1ee-a8a4f6625535 None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:01:00.275366 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e8fe61-bba6-4471-a1ee-a8a4f6625535 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:01:00.275913 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e8fe61-bba6-4471-a1ee-a8a4f6625535 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:01:00.280425 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0ee77b-b046-4ebd-af40-6b0468af7027 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:01:00.281158 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0ee77b-b046-4ebd-af40-6b0468af7027 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:01:00.282586 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0ee77b-b046-4ebd-af40-6b0468af7027 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:01:00.282876 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0ee77b-b046-4ebd-af40-6b0468af7027 admin admin] PATH_INFO: `/v3/projects/515055be6c664727847ab4eae19dd04f` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:01:00.284999 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0ee77b-b046-4ebd-af40-6b0468af7027 admin admin] RBAC: Authorizing `identity:delete_project(project_id=515055be6c664727847ab4eae19dd04f)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:01:00.288871 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0ee77b-b046-4ebd-af40-6b0468af7027 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:01:00.298875 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-f0e8fe61-bba6-4471-a1ee-a8a4f6625535 None None] MFA Rules not processed for user `e9fdf5c5f263474bb41ab41bdac6f78f`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:01:00.300215 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-f0e8fe61-bba6-4471-a1ee-a8a4f6625535 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:01:00.350363 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:01:00.373691 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-af0ee77b-b046-4ebd-af40-6b0468af7027 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '515055be6c664727847ab4eae19dd04f', 'request_id': 'req-af0ee77b-b046-4ebd-af40-6b0468af7027'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:01:00.382473 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1116/2255] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 16:01:00 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 16:01:00.389268 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-af0ee77b-b046-4ebd-af40-6b0468af7027 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '515055be6c664727847ab4eae19dd04f', 'request_id': 'req-af0ee77b-b046-4ebd-af40-6b0468af7027'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:01:00.401879 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-af0ee77b-b046-4ebd-af40-6b0468af7027 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '515055be6c664727847ab4eae19dd04f', 'request_id': 'req-af0ee77b-b046-4ebd-af40-6b0468af7027'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:01:00.417502 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-af0ee77b-b046-4ebd-af40-6b0468af7027 admin admin] The token cache is being invalidate because project 515055be6c664727847ab4eae19dd04f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81511) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 16:01:00.417800 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-af0ee77b-b046-4ebd-af40-6b0468af7027 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:01:00.421989 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-af0ee77b-b046-4ebd-af40-6b0468af7027 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:01:00.431137 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1140/2256] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Aug 30 16:01:00 2022] DELETE /identity/v3/projects/515055be6c664727847ab4eae19dd04f => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 16:01:00.675315 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e8fe61-bba6-4471-a1ee-a8a4f6625535 None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:01:00.677331 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-f0e8fe61-bba6-4471-a1ee-a8a4f6625535 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:01:00.691694 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e8fe61-bba6-4471-a1ee-a8a4f6625535 None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:01:00.751857 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39662e2-9c16-4768-9ba2-4bccbe24c045 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:01:00.752572 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39662e2-9c16-4768-9ba2-4bccbe24c045 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:01:00.752772 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39662e2-9c16-4768-9ba2-4bccbe24c045 admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:01:00.753047 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39662e2-9c16-4768-9ba2-4bccbe24c045 admin admin] PATH_INFO: `/v3/projects/26bb6b7007af4fdda66f640d3f258927` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:01:00.754252 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39662e2-9c16-4768-9ba2-4bccbe24c045 admin admin] RBAC: Authorizing `identity:delete_project(project_id=26bb6b7007af4fdda66f640d3f258927)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:01:00.756270 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39662e2-9c16-4768-9ba2-4bccbe24c045 admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:01:00.801498 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:01:00.809131 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-b39662e2-9c16-4768-9ba2-4bccbe24c045 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '26bb6b7007af4fdda66f640d3f258927', 'request_id': 'req-b39662e2-9c16-4768-9ba2-4bccbe24c045'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:01:00.818349 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-b39662e2-9c16-4768-9ba2-4bccbe24c045 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '26bb6b7007af4fdda66f640d3f258927', 'request_id': 'req-b39662e2-9c16-4768-9ba2-4bccbe24c045'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:01:00.824630 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-b39662e2-9c16-4768-9ba2-4bccbe24c045 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '26bb6b7007af4fdda66f640d3f258927', 'request_id': 'req-b39662e2-9c16-4768-9ba2-4bccbe24c045'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:01:00.835277 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-b39662e2-9c16-4768-9ba2-4bccbe24c045 admin admin] The token cache is being invalidate because project 26bb6b7007af4fdda66f640d3f258927 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81512) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 16:01:00.835572 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-b39662e2-9c16-4768-9ba2-4bccbe24c045 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:01:00.836226 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-b39662e2-9c16-4768-9ba2-4bccbe24c045 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:01:00.838003 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1117/2257] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Aug 30 16:01:00 2022] DELETE /identity/v3/projects/26bb6b7007af4fdda66f640d3f258927 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:01:00.841872 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0ee77b-b046-4ebd-af40-6b0468af7027 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:01:00.843331 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-af0ee77b-b046-4ebd-af40-6b0468af7027 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:01:00.856482 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0ee77b-b046-4ebd-af40-6b0468af7027 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:01:00.914972 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a759213-b953-4c26-b6e8-b95857e1ce88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:01:00.915620 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a759213-b953-4c26-b6e8-b95857e1ce88 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:01:00.915896 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a759213-b953-4c26-b6e8-b95857e1ce88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:01:00.916196 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a759213-b953-4c26-b6e8-b95857e1ce88 admin admin] PATH_INFO: `/v3/users/2bfce39a1e534209a9dcdae3a2683c7f` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:01:00.917374 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a759213-b953-4c26-b6e8-b95857e1ce88 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2bfce39a1e534209a9dcdae3a2683c7f)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:01:00.919308 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a759213-b953-4c26-b6e8-b95857e1ce88 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:01:00.970673 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-0a759213-b953-4c26-b6e8-b95857e1ce88 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2bfce39a1e534209a9dcdae3a2683c7f'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:01:00.982890 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-0a759213-b953-4c26-b6e8-b95857e1ce88 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2bfce39a1e534209a9dcdae3a2683c7f'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:01:00.992590 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-0a759213-b953-4c26-b6e8-b95857e1ce88 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2bfce39a1e534209a9dcdae3a2683c7f'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:01:00.993202 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-0a759213-b953-4c26-b6e8-b95857e1ce88 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2bfce39a1e534209a9dcdae3a2683c7f'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:01:01.002677 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-0a759213-b953-4c26-b6e8-b95857e1ce88 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2bfce39a1e534209a9dcdae3a2683c7f'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:01:01.008716 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1141/2258] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Aug 30 16:01:00 2022] DELETE /identity/v3/users/2bfce39a1e534209a9dcdae3a2683c7f => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:01:01.380595 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39662e2-9c16-4768-9ba2-4bccbe24c045 admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:01:01.382244 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-b39662e2-9c16-4768-9ba2-4bccbe24c045 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:01:01.396158 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39662e2-9c16-4768-9ba2-4bccbe24c045 admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:01:01.473867 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7294797d-e0e4-4475-952c-a6706ef345a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:01:01.474497 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7294797d-e0e4-4475-952c-a6706ef345a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:01:01.474771 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7294797d-e0e4-4475-952c-a6706ef345a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:01:01.475035 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7294797d-e0e4-4475-952c-a6706ef345a8 admin admin] PATH_INFO: `/v3/projects/8bd4ac278cba4b7c85bd0930cb03f7cb` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:01:01.476143 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7294797d-e0e4-4475-952c-a6706ef345a8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8bd4ac278cba4b7c85bd0930cb03f7cb)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:01:01.478110 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7294797d-e0e4-4475-952c-a6706ef345a8 admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:01:01.525642 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-7294797d-e0e4-4475-952c-a6706ef345a8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8bd4ac278cba4b7c85bd0930cb03f7cb', 'request_id': 'req-7294797d-e0e4-4475-952c-a6706ef345a8'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:01:01.535545 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-7294797d-e0e4-4475-952c-a6706ef345a8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8bd4ac278cba4b7c85bd0930cb03f7cb', 'request_id': 'req-7294797d-e0e4-4475-952c-a6706ef345a8'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:01:01.544292 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-7294797d-e0e4-4475-952c-a6706ef345a8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8bd4ac278cba4b7c85bd0930cb03f7cb', 'request_id': 'req-7294797d-e0e4-4475-952c-a6706ef345a8'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:01:01.556013 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-7294797d-e0e4-4475-952c-a6706ef345a8 admin admin] The token cache is being invalidate because project 8bd4ac278cba4b7c85bd0930cb03f7cb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81512) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 16:01:01.556309 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-7294797d-e0e4-4475-952c-a6706ef345a8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:01:01.557652 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-7294797d-e0e4-4475-952c-a6706ef345a8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:01:01.559320 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1118/2259] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Aug 30 16:01:01 2022] DELETE /identity/v3/projects/8bd4ac278cba4b7c85bd0930cb03f7cb => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:01:01.564292 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a759213-b953-4c26-b6e8-b95857e1ce88 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:01:01.566252 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-0a759213-b953-4c26-b6e8-b95857e1ce88 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:01:01.579204 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a759213-b953-4c26-b6e8-b95857e1ce88 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:01:01.623401 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:01:01.640877 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea5abe2-279a-4373-9818-1717dc32026f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:01:01.641560 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea5abe2-279a-4373-9818-1717dc32026f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:01:01.641954 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea5abe2-279a-4373-9818-1717dc32026f admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:01:01.642229 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea5abe2-279a-4373-9818-1717dc32026f admin admin] PATH_INFO: `/v3/users/441a522c485d4d0ca79c6fcc167bc8ba` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:01:01.643444 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea5abe2-279a-4373-9818-1717dc32026f admin admin] RBAC: Authorizing `identity:delete_user(user_id=441a522c485d4d0ca79c6fcc167bc8ba)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:01:01.645570 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea5abe2-279a-4373-9818-1717dc32026f admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:01:01.710273 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-cea5abe2-279a-4373-9818-1717dc32026f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '441a522c485d4d0ca79c6fcc167bc8ba'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:01:01.724447 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-cea5abe2-279a-4373-9818-1717dc32026f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '441a522c485d4d0ca79c6fcc167bc8ba'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:01:01.736227 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-cea5abe2-279a-4373-9818-1717dc32026f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '441a522c485d4d0ca79c6fcc167bc8ba'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:01:01.736849 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-cea5abe2-279a-4373-9818-1717dc32026f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '441a522c485d4d0ca79c6fcc167bc8ba'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:01:01.747654 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-cea5abe2-279a-4373-9818-1717dc32026f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '441a522c485d4d0ca79c6fcc167bc8ba'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:01:01.753685 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1142/2260] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Aug 30 16:01:01 2022] DELETE /identity/v3/users/441a522c485d4d0ca79c6fcc167bc8ba => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:01:02.054411 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7294797d-e0e4-4475-952c-a6706ef345a8 admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:01:02.055939 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-7294797d-e0e4-4475-952c-a6706ef345a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:01:02.068820 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7294797d-e0e4-4475-952c-a6706ef345a8 admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:01:02.136228 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:01:02.144266 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b1e4b9-5eec-4af6-be04-644b2d5aa356 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:01:02.144882 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b1e4b9-5eec-4af6-be04-644b2d5aa356 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:01:02.145393 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b1e4b9-5eec-4af6-be04-644b2d5aa356 admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:01:02.146060 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b1e4b9-5eec-4af6-be04-644b2d5aa356 admin admin] PATH_INFO: `/v3/projects/12b9adbe577c4a698ed317b9e6f942cb` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:01:02.147582 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b1e4b9-5eec-4af6-be04-644b2d5aa356 admin admin] RBAC: Authorizing `identity:delete_project(project_id=12b9adbe577c4a698ed317b9e6f942cb)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:01:02.149994 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b1e4b9-5eec-4af6-be04-644b2d5aa356 admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:01:02.201984 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-30b1e4b9-5eec-4af6-be04-644b2d5aa356 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '12b9adbe577c4a698ed317b9e6f942cb', 'request_id': 'req-30b1e4b9-5eec-4af6-be04-644b2d5aa356'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:01:02.211913 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-30b1e4b9-5eec-4af6-be04-644b2d5aa356 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '12b9adbe577c4a698ed317b9e6f942cb', 'request_id': 'req-30b1e4b9-5eec-4af6-be04-644b2d5aa356'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:01:02.218886 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-30b1e4b9-5eec-4af6-be04-644b2d5aa356 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '12b9adbe577c4a698ed317b9e6f942cb', 'request_id': 'req-30b1e4b9-5eec-4af6-be04-644b2d5aa356'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:01:02.230209 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-30b1e4b9-5eec-4af6-be04-644b2d5aa356 admin admin] The token cache is being invalidate because project 12b9adbe577c4a698ed317b9e6f942cb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81512) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 16:01:02.230491 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-30b1e4b9-5eec-4af6-be04-644b2d5aa356 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:01:02.231130 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-30b1e4b9-5eec-4af6-be04-644b2d5aa356 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:01:02.232664 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1119/2261] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Aug 30 16:01:02 2022] DELETE /identity/v3/projects/12b9adbe577c4a698ed317b9e6f942cb => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:01:02.238852 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab97512d-7023-4ab1-a06b-95b137123db0 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:01:02.239389 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab97512d-7023-4ab1-a06b-95b137123db0 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:01:02.239526 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab97512d-7023-4ab1-a06b-95b137123db0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:01:02.260429 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-ab97512d-7023-4ab1-a06b-95b137123db0 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:01:02.261607 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-ab97512d-7023-4ab1-a06b-95b137123db0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:01:02.327674 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1143/2262] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 16:01:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 16:01:02.339398 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fdf28e-992d-496e-9fb7-1d1cd85bdb9c None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:01:02.339760 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fdf28e-992d-496e-9fb7-1d1cd85bdb9c None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:01:02.340136 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fdf28e-992d-496e-9fb7-1d1cd85bdb9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:01:02.363209 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-88fdf28e-992d-496e-9fb7-1d1cd85bdb9c None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:01:02.364419 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-88fdf28e-992d-496e-9fb7-1d1cd85bdb9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:01:02.373191 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1120/2263] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 16:01:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:01:02.377691 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab97512d-7023-4ab1-a06b-95b137123db0 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:01:02.387655 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab97512d-7023-4ab1-a06b-95b137123db0 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:01:02.392893 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851a8c41-9ab6-42ae-b4b1-54a5a5b23856 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:01:02.393534 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851a8c41-9ab6-42ae-b4b1-54a5a5b23856 admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:01:02.393821 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851a8c41-9ab6-42ae-b4b1-54a5a5b23856 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:01:02.394094 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851a8c41-9ab6-42ae-b4b1-54a5a5b23856 admin admin] PATH_INFO: `/v3/domains` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:01:02.394798 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851a8c41-9ab6-42ae-b4b1-54a5a5b23856 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:01:02.395043 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851a8c41-9ab6-42ae-b4b1-54a5a5b23856 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:01:02.396695 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851a8c41-9ab6-42ae-b4b1-54a5a5b23856 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:01:02.408498 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1144/2264] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Aug 30 16:01:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:01:02.412887 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fdf28e-992d-496e-9fb7-1d1cd85bdb9c None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:01:02.422560 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fdf28e-992d-496e-9fb7-1d1cd85bdb9c None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:01:02.427711 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53c0a4e-c807-4616-8e08-3649c5acfff8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:01:02.428296 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53c0a4e-c807-4616-8e08-3649c5acfff8 admin admin] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:01:02.428572 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53c0a4e-c807-4616-8e08-3649c5acfff8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:01:02.428831 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53c0a4e-c807-4616-8e08-3649c5acfff8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:01:02.429490 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53c0a4e-c807-4616-8e08-3649c5acfff8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:01:02.431387 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53c0a4e-c807-4616-8e08-3649c5acfff8 admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:01:02.465055 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1121/2265] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Aug 30 16:01:02 2022] POST /identity/v3/projects => generated 375 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:01:02.469296 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851a8c41-9ab6-42ae-b4b1-54a5a5b23856 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:01:02.488233 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851a8c41-9ab6-42ae-b4b1-54a5a5b23856 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:01:02.540657 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:01:02.552190 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab48c6b4-4fa8-4a2b-a930-965178161ab5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:01:02.552925 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab48c6b4-4fa8-4a2b-a930-965178161ab5 admin admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:01:02.553295 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab48c6b4-4fa8-4a2b-a930-965178161ab5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:01:02.553893 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab48c6b4-4fa8-4a2b-a930-965178161ab5 admin admin] PATH_INFO: `/v3/users` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:01:02.554536 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab48c6b4-4fa8-4a2b-a930-965178161ab5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:01:02.556461 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab48c6b4-4fa8-4a2b-a930-965178161ab5 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:01:02.578625 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1145/2266] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Aug 30 16:01:02 2022] POST /identity/v3/users => generated 288 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:01:02.582907 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53c0a4e-c807-4616-8e08-3649c5acfff8 admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:01:02.594464 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53c0a4e-c807-4616-8e08-3649c5acfff8 admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:01:02.600048 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac5057d-7b9b-4a2f-aef6-d7fd34665f9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:01:02.600665 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac5057d-7b9b-4a2f-aef6-d7fd34665f9c admin admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:01:02.600938 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac5057d-7b9b-4a2f-aef6-d7fd34665f9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:01:02.601235 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac5057d-7b9b-4a2f-aef6-d7fd34665f9c admin admin] PATH_INFO: `/v3/roles` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:01:02.602042 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac5057d-7b9b-4a2f-aef6-d7fd34665f9c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:01:02.604496 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac5057d-7b9b-4a2f-aef6-d7fd34665f9c admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:01:02.616052 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1122/2267] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 30 16:01:02 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 16:01:02.620770 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab48c6b4-4fa8-4a2b-a930-965178161ab5 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:01:02.631274 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab48c6b4-4fa8-4a2b-a930-965178161ab5 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:01:02.636778 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba867a54-c4e6-47dc-a585-eec869438562 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:01:02.637547 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba867a54-c4e6-47dc-a585-eec869438562 admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:01:02.637886 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba867a54-c4e6-47dc-a585-eec869438562 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:01:02.638233 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba867a54-c4e6-47dc-a585-eec869438562 admin admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:01:02.645266 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba867a54-c4e6-47dc-a585-eec869438562 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:01:02.645266 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba867a54-c4e6-47dc-a585-eec869438562 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:01:02.653978 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1146/2268] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 30 16:01:02 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:01:02.658615 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac5057d-7b9b-4a2f-aef6-d7fd34665f9c admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:01:02.668933 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac5057d-7b9b-4a2f-aef6-d7fd34665f9c admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:01:02.673739 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8a058d-f256-4269-9d9d-d8707b52bd85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:01:02.674530 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8a058d-f256-4269-9d9d-d8707b52bd85 admin admin] REQUEST_METHOD: `PUT` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:01:02.675799 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8a058d-f256-4269-9d9d-d8707b52bd85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:01:02.676140 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8a058d-f256-4269-9d9d-d8707b52bd85 admin admin] PATH_INFO: `/v3/projects/dfcab2a556e24362bd92f0e1056c2007/users/0b6dd369313d43cbbba41af9876118b2/roles/df8f9f844b1b439181d75a07c1a93418` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:01:02.693957 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8a058d-f256-4269-9d9d-d8707b52bd85 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dfcab2a556e24362bd92f0e1056c2007, user_id=0b6dd369313d43cbbba41af9876118b2, role_id=df8f9f844b1b439181d75a07c1a93418)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:01:02.696929 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8a058d-f256-4269-9d9d-d8707b52bd85 admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:01:02.710514 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1123/2269] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 30 16:01:02 2022] PUT /identity/v3/projects/dfcab2a556e24362bd92f0e1056c2007/users/0b6dd369313d43cbbba41af9876118b2/roles/df8f9f844b1b439181d75a07c1a93418 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 16:01:02.722605 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad866ec-d343-48b4-a67f-1dcd953e5d12 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:01:02.722888 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad866ec-d343-48b4-a67f-1dcd953e5d12 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:01:02.723153 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad866ec-d343-48b4-a67f-1dcd953e5d12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:01:02.753953 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-dad866ec-d343-48b4-a67f-1dcd953e5d12 None None] MFA Rules not processed for user `0b6dd369313d43cbbba41af9876118b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:01:02.755056 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-dad866ec-d343-48b4-a67f-1dcd953e5d12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:01:02.831288 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1147/2270] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 16:01:02 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:01:02.842291 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8a058d-f256-4269-9d9d-d8707b52bd85 admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:01:02.844253 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-1b8a058d-f256-4269-9d9d-d8707b52bd85 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:01:02.904701 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8a058d-f256-4269-9d9d-d8707b52bd85 admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:01:02.908296 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf65e6b4-c9d9-495f-beab-fd94e0e2ca4c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0100a04821804993866ea20f159663f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:01:02.908907 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf65e6b4-c9d9-495f-beab-fd94e0e2ca4c service cinder] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:01:02.909183 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf65e6b4-c9d9-495f-beab-fd94e0e2ca4c service cinder] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:01:02.909480 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf65e6b4-c9d9-495f-beab-fd94e0e2ca4c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:01:02.917708 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf65e6b4-c9d9-495f-beab-fd94e0e2ca4c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:01:02.919151 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf65e6b4-c9d9-495f-beab-fd94e0e2ca4c service cinder] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:01:02.927885 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1124/2271] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 16:01:02 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:01:07.859891 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad866ec-d343-48b4-a67f-1dcd953e5d12 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:01:07.861413 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-dad866ec-d343-48b4-a67f-1dcd953e5d12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:01:07.939647 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad866ec-d343-48b4-a67f-1dcd953e5d12 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:01:07.948035 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9a8c7e-9a94-4e89-bf06-4a804f0bdc92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:01:07.948694 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9a8c7e-9a94-4e89-bf06-4a804f0bdc92 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:01:07.948974 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9a8c7e-9a94-4e89-bf06-4a804f0bdc92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:01:07.949283 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9a8c7e-9a94-4e89-bf06-4a804f0bdc92 admin admin] PATH_INFO: `/v3/users/bf02d15ca65a46fda47363c87eeaceaf` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:01:07.950457 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9a8c7e-9a94-4e89-bf06-4a804f0bdc92 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bf02d15ca65a46fda47363c87eeaceaf)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:01:07.952463 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9a8c7e-9a94-4e89-bf06-4a804f0bdc92 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:01:08.028492 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-eb9a8c7e-9a94-4e89-bf06-4a804f0bdc92 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bf02d15ca65a46fda47363c87eeaceaf'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:01:08.061653 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=89,keystone:INSERT=3,keystone:DELETE=8 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:01:08.067844 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-eb9a8c7e-9a94-4e89-bf06-4a804f0bdc92 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bf02d15ca65a46fda47363c87eeaceaf'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:01:08.081034 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-eb9a8c7e-9a94-4e89-bf06-4a804f0bdc92 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bf02d15ca65a46fda47363c87eeaceaf'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:01:08.081838 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-eb9a8c7e-9a94-4e89-bf06-4a804f0bdc92 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bf02d15ca65a46fda47363c87eeaceaf'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:01:08.099428 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-eb9a8c7e-9a94-4e89-bf06-4a804f0bdc92 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bf02d15ca65a46fda47363c87eeaceaf'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:01:08.108774 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1148/2272] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Aug 30 16:01:07 2022] DELETE /identity/v3/users/bf02d15ca65a46fda47363c87eeaceaf => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:01:08.114015 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76dd47f-a6c3-42ab-ad2a-c6fc7c8b6f34 None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:01:08.114545 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76dd47f-a6c3-42ab-ad2a-c6fc7c8b6f34 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:01:08.114545 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76dd47f-a6c3-42ab-ad2a-c6fc7c8b6f34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:01:08.144533 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-f76dd47f-a6c3-42ab-ad2a-c6fc7c8b6f34 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:01:08.146241 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-f76dd47f-a6c3-42ab-ad2a-c6fc7c8b6f34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:01:08.211035 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:01:08.302580 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1125/2273] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 16:01:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:01:08.345437 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9a8c7e-9a94-4e89-bf06-4a804f0bdc92 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:01:08.349973 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-eb9a8c7e-9a94-4e89-bf06-4a804f0bdc92 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:01:08.514385 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9a8c7e-9a94-4e89-bf06-4a804f0bdc92 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:01:08.522669 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbde0a0f-be8c-409b-a779-0fff32f54714 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f4157105f3488f807b7475008ab722', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:01:08.525184 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbde0a0f-be8c-409b-a779-0fff32f54714 service neutron] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:01:08.525529 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbde0a0f-be8c-409b-a779-0fff32f54714 service neutron] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:01:08.525801 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbde0a0f-be8c-409b-a779-0fff32f54714 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:01:08.556471 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbde0a0f-be8c-409b-a779-0fff32f54714 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:01:08.558312 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbde0a0f-be8c-409b-a779-0fff32f54714 service neutron] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:01:08.573065 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1149/2274] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 16:01:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 228 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:01:08.958241 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76dd47f-a6c3-42ab-ad2a-c6fc7c8b6f34 None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:01:08.959814 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-f76dd47f-a6c3-42ab-ad2a-c6fc7c8b6f34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:01:08.972558 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76dd47f-a6c3-42ab-ad2a-c6fc7c8b6f34 None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:01:08.980562 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8e4394-0b14-47a1-bb71-84c8fe621627 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:01:08.981358 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8e4394-0b14-47a1-bb71-84c8fe621627 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:01:08.981727 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8e4394-0b14-47a1-bb71-84c8fe621627 admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:01:08.982094 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8e4394-0b14-47a1-bb71-84c8fe621627 admin admin] PATH_INFO: `/v3/projects/310e0bee3254457db6cfb45c50e82cf6` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:01:08.983349 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8e4394-0b14-47a1-bb71-84c8fe621627 admin admin] RBAC: Authorizing `identity:delete_project(project_id=310e0bee3254457db6cfb45c50e82cf6)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:01:08.985902 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8e4394-0b14-47a1-bb71-84c8fe621627 admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:01:09.038941 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-5a8e4394-0b14-47a1-bb71-84c8fe621627 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '310e0bee3254457db6cfb45c50e82cf6', 'request_id': 'req-5a8e4394-0b14-47a1-bb71-84c8fe621627'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:01:09.047084 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-5a8e4394-0b14-47a1-bb71-84c8fe621627 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '310e0bee3254457db6cfb45c50e82cf6', 'request_id': 'req-5a8e4394-0b14-47a1-bb71-84c8fe621627'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:01:09.053599 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-5a8e4394-0b14-47a1-bb71-84c8fe621627 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '310e0bee3254457db6cfb45c50e82cf6', 'request_id': 'req-5a8e4394-0b14-47a1-bb71-84c8fe621627'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:01:09.064270 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-5a8e4394-0b14-47a1-bb71-84c8fe621627 admin admin] The token cache is being invalidate because project 310e0bee3254457db6cfb45c50e82cf6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81512) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 16:01:09.064561 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-5a8e4394-0b14-47a1-bb71-84c8fe621627 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:01:09.065439 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-5a8e4394-0b14-47a1-bb71-84c8fe621627 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:01:09.066824 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1126/2275] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Aug 30 16:01:08 2022] DELETE /identity/v3/projects/310e0bee3254457db6cfb45c50e82cf6 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:01:09.071003 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbde0a0f-be8c-409b-a779-0fff32f54714 service neutron] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:01:09.085432 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbde0a0f-be8c-409b-a779-0fff32f54714 service neutron] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:01:09.143182 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30481249-ca36-4da8-849c-4a2165a64d25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:01:09.143844 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30481249-ca36-4da8-849c-4a2165a64d25 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:01:09.144123 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30481249-ca36-4da8-849c-4a2165a64d25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:01:09.144393 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30481249-ca36-4da8-849c-4a2165a64d25 admin admin] PATH_INFO: `/v3/users/277631c6432e4ad0a5f98f9052301fd7` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:01:09.146000 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30481249-ca36-4da8-849c-4a2165a64d25 admin admin] RBAC: Authorizing `identity:delete_user(user_id=277631c6432e4ad0a5f98f9052301fd7)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:01:09.149631 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30481249-ca36-4da8-849c-4a2165a64d25 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:01:09.204608 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-30481249-ca36-4da8-849c-4a2165a64d25 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '277631c6432e4ad0a5f98f9052301fd7'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:01:09.216200 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-30481249-ca36-4da8-849c-4a2165a64d25 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '277631c6432e4ad0a5f98f9052301fd7'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:01:09.224621 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-30481249-ca36-4da8-849c-4a2165a64d25 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '277631c6432e4ad0a5f98f9052301fd7'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:01:09.225547 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-30481249-ca36-4da8-849c-4a2165a64d25 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '277631c6432e4ad0a5f98f9052301fd7'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:01:09.236141 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-30481249-ca36-4da8-849c-4a2165a64d25 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '277631c6432e4ad0a5f98f9052301fd7'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:01:09.239008 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:01:09.244240 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1150/2276] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Aug 30 16:01:09 2022] DELETE /identity/v3/users/277631c6432e4ad0a5f98f9052301fd7 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:01:09.611388 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8e4394-0b14-47a1-bb71-84c8fe621627 admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:01:09.613323 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-5a8e4394-0b14-47a1-bb71-84c8fe621627 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:01:09.648165 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8e4394-0b14-47a1-bb71-84c8fe621627 admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:01:09.707150 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81d7952-1314-475c-80bd-e34758f23b74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:01:09.707780 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81d7952-1314-475c-80bd-e34758f23b74 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:01:09.708053 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81d7952-1314-475c-80bd-e34758f23b74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:01:09.708321 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81d7952-1314-475c-80bd-e34758f23b74 admin admin] PATH_INFO: `/v3/projects/be93ec67199443579907c5d948e925b8` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:01:09.709467 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81d7952-1314-475c-80bd-e34758f23b74 admin admin] RBAC: Authorizing `identity:delete_project(project_id=be93ec67199443579907c5d948e925b8)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:01:09.711375 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81d7952-1314-475c-80bd-e34758f23b74 admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:01:09.759219 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-d81d7952-1314-475c-80bd-e34758f23b74 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'be93ec67199443579907c5d948e925b8', 'request_id': 'req-d81d7952-1314-475c-80bd-e34758f23b74'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:01:09.761843 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:01:09.768754 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-d81d7952-1314-475c-80bd-e34758f23b74 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'be93ec67199443579907c5d948e925b8', 'request_id': 'req-d81d7952-1314-475c-80bd-e34758f23b74'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:01:09.775915 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-d81d7952-1314-475c-80bd-e34758f23b74 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'be93ec67199443579907c5d948e925b8', 'request_id': 'req-d81d7952-1314-475c-80bd-e34758f23b74'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:01:09.787179 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-d81d7952-1314-475c-80bd-e34758f23b74 admin admin] The token cache is being invalidate because project be93ec67199443579907c5d948e925b8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81512) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 16:01:09.787624 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-d81d7952-1314-475c-80bd-e34758f23b74 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:01:09.788556 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-d81d7952-1314-475c-80bd-e34758f23b74 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:01:09.790216 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1127/2277] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Aug 30 16:01:09 2022] DELETE /identity/v3/projects/be93ec67199443579907c5d948e925b8 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:01:09.795410 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44f06df-c299-4964-9f70-7666ba77d3c3 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:01:09.795825 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44f06df-c299-4964-9f70-7666ba77d3c3 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:01:09.796166 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44f06df-c299-4964-9f70-7666ba77d3c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:01:09.818285 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-a44f06df-c299-4964-9f70-7666ba77d3c3 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:01:09.819576 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-a44f06df-c299-4964-9f70-7666ba77d3c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:01:09.886453 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1151/2278] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 16:01:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:01:09.895027 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17db3b44-4fbb-4cd9-aaed-1029e628eb89 None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:01:09.895313 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17db3b44-4fbb-4cd9-aaed-1029e628eb89 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:01:09.895582 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17db3b44-4fbb-4cd9-aaed-1029e628eb89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:01:09.916258 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-17db3b44-4fbb-4cd9-aaed-1029e628eb89 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:01:09.917622 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-17db3b44-4fbb-4cd9-aaed-1029e628eb89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:01:09.926714 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1128/2279] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 16:01:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:01:09.931047 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44f06df-c299-4964-9f70-7666ba77d3c3 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:01:09.941451 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44f06df-c299-4964-9f70-7666ba77d3c3 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:01:09.946198 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e41d6e-09f7-464f-a995-59a7673f82bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:01:09.946819 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e41d6e-09f7-464f-a995-59a7673f82bb admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:01:09.947096 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e41d6e-09f7-464f-a995-59a7673f82bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:01:09.947363 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e41d6e-09f7-464f-a995-59a7673f82bb admin admin] PATH_INFO: `/v3/domains` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:01:09.948017 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e41d6e-09f7-464f-a995-59a7673f82bb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:01:09.948318 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e41d6e-09f7-464f-a995-59a7673f82bb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:01:09.949949 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e41d6e-09f7-464f-a995-59a7673f82bb admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:01:09.961998 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1152/2280] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Aug 30 16:01:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:01:09.966321 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17db3b44-4fbb-4cd9-aaed-1029e628eb89 None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:01:09.976217 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17db3b44-4fbb-4cd9-aaed-1029e628eb89 None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:01:09.981345 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af15aa99-6252-41d4-b036-1a23a339657c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:01:09.982093 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af15aa99-6252-41d4-b036-1a23a339657c admin admin] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:01:09.982376 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af15aa99-6252-41d4-b036-1a23a339657c admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:01:09.982647 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af15aa99-6252-41d4-b036-1a23a339657c admin admin] PATH_INFO: `/v3/projects` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:01:09.983284 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af15aa99-6252-41d4-b036-1a23a339657c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:01:09.985438 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af15aa99-6252-41d4-b036-1a23a339657c admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:01:10.019582 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1129/2281] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Aug 30 16:01:09 2022] POST /identity/v3/projects => generated 363 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:01:10.024115 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e41d6e-09f7-464f-a995-59a7673f82bb admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:01:10.034486 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e41d6e-09f7-464f-a995-59a7673f82bb admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:01:10.097022 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11081485-7f05-4b07-953b-1a7fdf246849 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:01:10.097773 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11081485-7f05-4b07-953b-1a7fdf246849 admin admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:01:10.097966 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11081485-7f05-4b07-953b-1a7fdf246849 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:01:10.098242 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11081485-7f05-4b07-953b-1a7fdf246849 admin admin] PATH_INFO: `/v3/users` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:01:10.098934 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11081485-7f05-4b07-953b-1a7fdf246849 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:01:10.100966 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11081485-7f05-4b07-953b-1a7fdf246849 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:01:10.126146 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1153/2282] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Aug 30 16:01:10 2022] POST /identity/v3/users => generated 282 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:01:10.130570 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af15aa99-6252-41d4-b036-1a23a339657c admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:01:10.141929 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af15aa99-6252-41d4-b036-1a23a339657c admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:01:10.146847 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd77be6b-5431-41aa-92db-af4a17302871 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:01:10.147534 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd77be6b-5431-41aa-92db-af4a17302871 admin admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:01:10.147841 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd77be6b-5431-41aa-92db-af4a17302871 admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:01:10.148110 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd77be6b-5431-41aa-92db-af4a17302871 admin admin] PATH_INFO: `/v3/roles` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:01:10.148964 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd77be6b-5431-41aa-92db-af4a17302871 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:01:10.150894 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd77be6b-5431-41aa-92db-af4a17302871 admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:01:10.169707 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1130/2283] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 30 16:01:10 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 16:01:10.171004 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11081485-7f05-4b07-953b-1a7fdf246849 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:01:10.181918 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11081485-7f05-4b07-953b-1a7fdf246849 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:01:10.187421 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2725c403-8434-4281-8725-b7f37d3608cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:01:10.188697 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2725c403-8434-4281-8725-b7f37d3608cd admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:01:10.189128 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2725c403-8434-4281-8725-b7f37d3608cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:01:10.189434 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2725c403-8434-4281-8725-b7f37d3608cd admin admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:01:10.190133 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2725c403-8434-4281-8725-b7f37d3608cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:01:10.191949 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2725c403-8434-4281-8725-b7f37d3608cd admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:01:10.203969 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1154/2284] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 30 16:01:10 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:01:10.208554 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd77be6b-5431-41aa-92db-af4a17302871 admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:01:10.219088 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd77be6b-5431-41aa-92db-af4a17302871 admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:01:10.224325 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245d0a35-5243-42e8-8a32-b459d3d9e832 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:01:10.225034 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245d0a35-5243-42e8-8a32-b459d3d9e832 admin admin] REQUEST_METHOD: `PUT` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:01:10.225345 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245d0a35-5243-42e8-8a32-b459d3d9e832 admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:01:10.225619 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245d0a35-5243-42e8-8a32-b459d3d9e832 admin admin] PATH_INFO: `/v3/projects/def3b95b29d04e3680f087590bfcc8a4/users/3028c6c8fbc443a1a1d4395961e173df/roles/df8f9f844b1b439181d75a07c1a93418` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:01:10.242645 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245d0a35-5243-42e8-8a32-b459d3d9e832 admin admin] RBAC: Authorizing `identity:create_grant(project_id=def3b95b29d04e3680f087590bfcc8a4, user_id=3028c6c8fbc443a1a1d4395961e173df, role_id=df8f9f844b1b439181d75a07c1a93418)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:01:10.245502 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245d0a35-5243-42e8-8a32-b459d3d9e832 admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:01:10.261326 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1131/2285] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 30 16:01:10 2022] PUT /identity/v3/projects/def3b95b29d04e3680f087590bfcc8a4/users/3028c6c8fbc443a1a1d4395961e173df/roles/df8f9f844b1b439181d75a07c1a93418 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:01:10.273087 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660ed791-c446-480e-ae48-0266249b6ce6 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:01:10.273403 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660ed791-c446-480e-ae48-0266249b6ce6 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:01:10.273677 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660ed791-c446-480e-ae48-0266249b6ce6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:01:10.302367 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-660ed791-c446-480e-ae48-0266249b6ce6 None None] MFA Rules not processed for user `3028c6c8fbc443a1a1d4395961e173df`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:01:10.303581 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-660ed791-c446-480e-ae48-0266249b6ce6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:01:10.364821 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:01:10.376624 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1155/2286] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 16:01:10 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:01:10.380607 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245d0a35-5243-42e8-8a32-b459d3d9e832 admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:01:10.454971 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245d0a35-5243-42e8-8a32-b459d3d9e832 admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:01:10.462171 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1560551-f668-414b-9815-3553136d287c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:01:10.462899 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1560551-f668-414b-9815-3553136d287c admin admin] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:01:10.463243 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1560551-f668-414b-9815-3553136d287c admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:01:10.463578 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1560551-f668-414b-9815-3553136d287c admin admin] PATH_INFO: `/v3/projects` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:01:10.464370 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1560551-f668-414b-9815-3553136d287c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:01:10.471659 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1560551-f668-414b-9815-3553136d287c admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:01:10.501128 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1132/2287] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Aug 30 16:01:10 2022] POST /identity/v3/projects => generated 365 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 16:01:10.502356 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660ed791-c446-480e-ae48-0266249b6ce6 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:01:10.513919 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660ed791-c446-480e-ae48-0266249b6ce6 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:01:10.582556 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e25411-6205-4db8-bf7d-3ba13857ba3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:01:10.582773 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e25411-6205-4db8-bf7d-3ba13857ba3e admin admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:01:10.583010 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e25411-6205-4db8-bf7d-3ba13857ba3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:01:10.583275 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e25411-6205-4db8-bf7d-3ba13857ba3e admin admin] PATH_INFO: `/v3/users` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:01:10.583945 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e25411-6205-4db8-bf7d-3ba13857ba3e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:01:10.586064 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e25411-6205-4db8-bf7d-3ba13857ba3e admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:01:10.613297 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1156/2288] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Aug 30 16:01:10 2022] POST /identity/v3/users => generated 289 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:01:10.617555 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1560551-f668-414b-9815-3553136d287c admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:01:10.629670 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1560551-f668-414b-9815-3553136d287c admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:01:10.635745 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe8c98a-9ace-4f15-bab4-9f66b1c37b3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:01:10.636441 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe8c98a-9ace-4f15-bab4-9f66b1c37b3e admin admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:01:10.636758 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe8c98a-9ace-4f15-bab4-9f66b1c37b3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:01:10.637086 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe8c98a-9ace-4f15-bab4-9f66b1c37b3e admin admin] PATH_INFO: `/v3/roles` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:01:10.637958 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe8c98a-9ace-4f15-bab4-9f66b1c37b3e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:01:10.639576 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe8c98a-9ace-4f15-bab4-9f66b1c37b3e admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:01:10.653340 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1133/2289] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 30 16:01:10 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:01:10.662547 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e25411-6205-4db8-bf7d-3ba13857ba3e admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:01:10.678824 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e25411-6205-4db8-bf7d-3ba13857ba3e admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:01:10.685052 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9933fe52-46d7-4b0d-89f1-bc983a383533 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:01:10.685940 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9933fe52-46d7-4b0d-89f1-bc983a383533 admin admin] REQUEST_METHOD: `PUT` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:01:10.686293 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9933fe52-46d7-4b0d-89f1-bc983a383533 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:01:10.686652 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9933fe52-46d7-4b0d-89f1-bc983a383533 admin admin] PATH_INFO: `/v3/projects/f2a878ad1d794e6f89ad23bc44c4b7fa/users/c9ca2b72efda42498a45d5ed2129fcbe/roles/f265386950a641ac8413b53eda4ae34d` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:01:10.705354 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9933fe52-46d7-4b0d-89f1-bc983a383533 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f2a878ad1d794e6f89ad23bc44c4b7fa, user_id=c9ca2b72efda42498a45d5ed2129fcbe, role_id=f265386950a641ac8413b53eda4ae34d)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:01:10.708266 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9933fe52-46d7-4b0d-89f1-bc983a383533 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:01:10.727328 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1157/2290] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 30 16:01:10 2022] PUT /identity/v3/projects/f2a878ad1d794e6f89ad23bc44c4b7fa/users/c9ca2b72efda42498a45d5ed2129fcbe/roles/f265386950a641ac8413b53eda4ae34d => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:01:10.731815 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe8c98a-9ace-4f15-bab4-9f66b1c37b3e admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:01:10.743815 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe8c98a-9ace-4f15-bab4-9f66b1c37b3e admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:01:10.801366 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:DELETE=3,keystone:SELECT=94,keystone:INSERT=3 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:01:10.822256 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581afafe-77c7-4f9e-ac98-ebd5e4a7d87c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:01:10.822882 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581afafe-77c7-4f9e-ac98-ebd5e4a7d87c admin admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:01:10.823158 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581afafe-77c7-4f9e-ac98-ebd5e4a7d87c admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:01:10.823425 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581afafe-77c7-4f9e-ac98-ebd5e4a7d87c admin admin] PATH_INFO: `/v3/roles` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:01:10.824067 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581afafe-77c7-4f9e-ac98-ebd5e4a7d87c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:01:10.825912 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581afafe-77c7-4f9e-ac98-ebd5e4a7d87c admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:01:10.836932 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1134/2291] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 30 16:01:10 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:01:10.841172 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9933fe52-46d7-4b0d-89f1-bc983a383533 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:01:10.851740 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9933fe52-46d7-4b0d-89f1-bc983a383533 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:01:10.859060 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a928e29-eed8-4078-aac6-ba8c85029f9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:01:10.859881 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a928e29-eed8-4078-aac6-ba8c85029f9f admin admin] REQUEST_METHOD: `PUT` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:01:10.860557 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a928e29-eed8-4078-aac6-ba8c85029f9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:01:10.860866 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a928e29-eed8-4078-aac6-ba8c85029f9f admin admin] PATH_INFO: `/v3/projects/f2a878ad1d794e6f89ad23bc44c4b7fa/users/c9ca2b72efda42498a45d5ed2129fcbe/roles/f265386950a641ac8413b53eda4ae34d` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:01:10.864480 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a928e29-eed8-4078-aac6-ba8c85029f9f admin admin] RBAC: Authorizing `identity:create_grant(project_id=f2a878ad1d794e6f89ad23bc44c4b7fa, user_id=c9ca2b72efda42498a45d5ed2129fcbe, role_id=f265386950a641ac8413b53eda4ae34d)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:01:10.869097 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a928e29-eed8-4078-aac6-ba8c85029f9f admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:01:10.881866 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1158/2292] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 30 16:01:10 2022] PUT /identity/v3/projects/f2a878ad1d794e6f89ad23bc44c4b7fa/users/c9ca2b72efda42498a45d5ed2129fcbe/roles/f265386950a641ac8413b53eda4ae34d => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:01:10.893695 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8880ed82-8d9a-453a-9060-806b4291fbdf None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:01:10.893863 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8880ed82-8d9a-453a-9060-806b4291fbdf None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:01:10.894675 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8880ed82-8d9a-453a-9060-806b4291fbdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:01:10.926341 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-8880ed82-8d9a-453a-9060-806b4291fbdf None None] MFA Rules not processed for user `c9ca2b72efda42498a45d5ed2129fcbe`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:01:10.928161 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-8880ed82-8d9a-453a-9060-806b4291fbdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:01:11.009570 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1135/2293] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 16:01:10 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:01:11.022012 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a928e29-eed8-4078-aac6-ba8c85029f9f admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:01:11.024025 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-4a928e29-eed8-4078-aac6-ba8c85029f9f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:01:11.085967 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a928e29-eed8-4078-aac6-ba8c85029f9f admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:01:11.089783 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3913f2-4826-48d4-af20-f3fc70fc7f31 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0100a04821804993866ea20f159663f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:01:11.090388 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3913f2-4826-48d4-af20-f3fc70fc7f31 service cinder] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:01:11.090668 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3913f2-4826-48d4-af20-f3fc70fc7f31 service cinder] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:01:11.090937 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3913f2-4826-48d4-af20-f3fc70fc7f31 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:01:11.099850 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3913f2-4826-48d4-af20-f3fc70fc7f31 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:01:11.101326 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3913f2-4826-48d4-af20-f3fc70fc7f31 service cinder] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:01:11.110843 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1159/2294] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 16:01:11 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:01:11.190447 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8880ed82-8d9a-453a-9060-806b4291fbdf None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:01:11.200147 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8880ed82-8d9a-453a-9060-806b4291fbdf None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:01:11.203966 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0718069b-3e88-49d2-8f17-51a9df2339b8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0100a04821804993866ea20f159663f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:01:11.204565 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0718069b-3e88-49d2-8f17-51a9df2339b8 service cinder] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:01:11.204842 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0718069b-3e88-49d2-8f17-51a9df2339b8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:01:11.205111 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0718069b-3e88-49d2-8f17-51a9df2339b8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:01:11.266826 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0718069b-3e88-49d2-8f17-51a9df2339b8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:01:11.268280 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0718069b-3e88-49d2-8f17-51a9df2339b8 service cinder] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:01:11.278432 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1136/2295] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 16:01:11 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 16:01:14.081786 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3913f2-4826-48d4-af20-f3fc70fc7f31 service cinder] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:01:14.083372 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-de3913f2-4826-48d4-af20-f3fc70fc7f31 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:01:14.170873 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:01:14.185196 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3913f2-4826-48d4-af20-f3fc70fc7f31 service cinder] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:01:14.193338 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bed52f9-a633-4635-8c43-ebb29676ac8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:01:14.194071 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bed52f9-a633-4635-8c43-ebb29676ac8e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:01:14.194257 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bed52f9-a633-4635-8c43-ebb29676ac8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:01:14.194683 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bed52f9-a633-4635-8c43-ebb29676ac8e admin admin] PATH_INFO: `/v3/users/0b6dd369313d43cbbba41af9876118b2` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:01:14.196174 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bed52f9-a633-4635-8c43-ebb29676ac8e admin admin] RBAC: Authorizing `identity:delete_user(user_id=0b6dd369313d43cbbba41af9876118b2)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:01:14.198572 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bed52f9-a633-4635-8c43-ebb29676ac8e admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:01:14.268569 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-2bed52f9-a633-4635-8c43-ebb29676ac8e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0b6dd369313d43cbbba41af9876118b2'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:01:14.282658 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-2bed52f9-a633-4635-8c43-ebb29676ac8e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0b6dd369313d43cbbba41af9876118b2'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:01:14.292637 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-2bed52f9-a633-4635-8c43-ebb29676ac8e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0b6dd369313d43cbbba41af9876118b2'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:01:14.293650 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-2bed52f9-a633-4635-8c43-ebb29676ac8e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0b6dd369313d43cbbba41af9876118b2'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:01:14.302127 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-2bed52f9-a633-4635-8c43-ebb29676ac8e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0b6dd369313d43cbbba41af9876118b2'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:01:14.309337 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1160/2296] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Aug 30 16:01:14 2022] DELETE /identity/v3/users/0b6dd369313d43cbbba41af9876118b2 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:01:14.313880 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b508a12-bbcb-4b1f-b86d-c3e82b27d4c9 None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:01:14.314390 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b508a12-bbcb-4b1f-b86d-c3e82b27d4c9 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:01:14.314475 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b508a12-bbcb-4b1f-b86d-c3e82b27d4c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:01:14.336082 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-0b508a12-bbcb-4b1f-b86d-c3e82b27d4c9 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:01:14.337412 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-0b508a12-bbcb-4b1f-b86d-c3e82b27d4c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:01:14.413596 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1137/2297] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 16:01:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:01:14.418754 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bed52f9-a633-4635-8c43-ebb29676ac8e admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:01:14.420201 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-2bed52f9-a633-4635-8c43-ebb29676ac8e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:01:14.490770 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bed52f9-a633-4635-8c43-ebb29676ac8e admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:01:14.495396 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b45d6c-29f6-4ba5-bdda-1c6c8a18513d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f4157105f3488f807b7475008ab722', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:01:14.496020 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b45d6c-29f6-4ba5-bdda-1c6c8a18513d service neutron] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:01:14.497386 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b45d6c-29f6-4ba5-bdda-1c6c8a18513d service neutron] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:01:14.497386 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b45d6c-29f6-4ba5-bdda-1c6c8a18513d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:01:14.511872 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b45d6c-29f6-4ba5-bdda-1c6c8a18513d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:01:14.513377 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b45d6c-29f6-4ba5-bdda-1c6c8a18513d service neutron] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:01:14.524502 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1161/2298] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 16:01:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:01:14.666871 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf2d108-68a4-4bd3-8fae-ed49a684ac8d None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:01:14.667162 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf2d108-68a4-4bd3-8fae-ed49a684ac8d None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:01:14.667440 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf2d108-68a4-4bd3-8fae-ed49a684ac8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:01:14.711876 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-acf2d108-68a4-4bd3-8fae-ed49a684ac8d None None] MFA Rules not processed for user `b133ae684f20468f982ae0c25020d837`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:01:14.727430 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-acf2d108-68a4-4bd3-8fae-ed49a684ac8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:01:14.734556 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:01:14.802463 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1138/2299] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 16:01:14 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:01:14.940406 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b45d6c-29f6-4ba5-bdda-1c6c8a18513d service neutron] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:01:14.941966 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-b1b45d6c-29f6-4ba5-bdda-1c6c8a18513d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:01:14.970600 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b45d6c-29f6-4ba5-bdda-1c6c8a18513d service neutron] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:01:14.975807 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a88d96-4ff7-4c71-9ddb-8580b0738620 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:01:14.976809 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a88d96-4ff7-4c71-9ddb-8580b0738620 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:01:14.977088 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a88d96-4ff7-4c71-9ddb-8580b0738620 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:01:14.977493 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a88d96-4ff7-4c71-9ddb-8580b0738620 admin admin] PATH_INFO: `/v3/projects/dfcab2a556e24362bd92f0e1056c2007` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:01:14.978864 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a88d96-4ff7-4c71-9ddb-8580b0738620 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dfcab2a556e24362bd92f0e1056c2007)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:01:14.980806 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a88d96-4ff7-4c71-9ddb-8580b0738620 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:01:15.034974 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-51a88d96-4ff7-4c71-9ddb-8580b0738620 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dfcab2a556e24362bd92f0e1056c2007', 'request_id': 'req-51a88d96-4ff7-4c71-9ddb-8580b0738620'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:01:15.045489 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-51a88d96-4ff7-4c71-9ddb-8580b0738620 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dfcab2a556e24362bd92f0e1056c2007', 'request_id': 'req-51a88d96-4ff7-4c71-9ddb-8580b0738620'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:01:15.052134 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-51a88d96-4ff7-4c71-9ddb-8580b0738620 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dfcab2a556e24362bd92f0e1056c2007', 'request_id': 'req-51a88d96-4ff7-4c71-9ddb-8580b0738620'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:01:15.056681 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:01:15.063540 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-51a88d96-4ff7-4c71-9ddb-8580b0738620 admin admin] The token cache is being invalidate because project dfcab2a556e24362bd92f0e1056c2007 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81511) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 16:01:15.064147 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-51a88d96-4ff7-4c71-9ddb-8580b0738620 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:01:15.065524 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-51a88d96-4ff7-4c71-9ddb-8580b0738620 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:01:15.067049 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1162/2300] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Aug 30 16:01:14 2022] DELETE /identity/v3/projects/dfcab2a556e24362bd92f0e1056c2007 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:01:15.071966 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41049cd2-802a-45a3-93a4-867c6b6fcc67 None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:01:15.072251 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41049cd2-802a-45a3-93a4-867c6b6fcc67 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:01:15.072519 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41049cd2-802a-45a3-93a4-867c6b6fcc67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:01:15.093242 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-41049cd2-802a-45a3-93a4-867c6b6fcc67 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:01:15.094619 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-41049cd2-802a-45a3-93a4-867c6b6fcc67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:01:15.157196 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1139/2301] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 16:01:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:01:15.165204 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cddd05-dc1c-4b2d-9892-6dbcc10b2ad9 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:01:15.165763 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cddd05-dc1c-4b2d-9892-6dbcc10b2ad9 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:01:15.165763 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cddd05-dc1c-4b2d-9892-6dbcc10b2ad9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:01:15.187028 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-72cddd05-dc1c-4b2d-9892-6dbcc10b2ad9 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:01:15.188521 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-72cddd05-dc1c-4b2d-9892-6dbcc10b2ad9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:01:15.198362 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1163/2302] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 16:01:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:01:15.202641 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41049cd2-802a-45a3-93a4-867c6b6fcc67 None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:01:15.214120 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41049cd2-802a-45a3-93a4-867c6b6fcc67 None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:01:15.218901 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f251d1b2-f552-4f45-9a0c-74118b964527 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:01:15.219526 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f251d1b2-f552-4f45-9a0c-74118b964527 admin admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:01:15.219838 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f251d1b2-f552-4f45-9a0c-74118b964527 admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:01:15.220135 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f251d1b2-f552-4f45-9a0c-74118b964527 admin admin] PATH_INFO: `/v3/domains` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:01:15.220814 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f251d1b2-f552-4f45-9a0c-74118b964527 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81512) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:01:15.221192 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f251d1b2-f552-4f45-9a0c-74118b964527 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:01:15.222852 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f251d1b2-f552-4f45-9a0c-74118b964527 admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:01:15.237921 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1140/2303] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Aug 30 16:01:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:01:15.241994 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cddd05-dc1c-4b2d-9892-6dbcc10b2ad9 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:01:15.251785 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cddd05-dc1c-4b2d-9892-6dbcc10b2ad9 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:01:15.256495 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72692b4-758a-4dc0-81f5-4be446518823 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:01:15.257135 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72692b4-758a-4dc0-81f5-4be446518823 admin admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:01:15.257453 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72692b4-758a-4dc0-81f5-4be446518823 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:01:15.258464 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72692b4-758a-4dc0-81f5-4be446518823 admin admin] PATH_INFO: `/v3/projects` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:01:15.258464 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72692b4-758a-4dc0-81f5-4be446518823 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:01:15.260290 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72692b4-758a-4dc0-81f5-4be446518823 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:01:15.289991 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1164/2304] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Aug 30 16:01:15 2022] POST /identity/v3/projects => generated 397 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 16:01:15.294155 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f251d1b2-f552-4f45-9a0c-74118b964527 admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:01:15.304802 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f251d1b2-f552-4f45-9a0c-74118b964527 admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:01:15.364491 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbed5a8c-7271-4749-b8b3-d1f8ba29bd8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:01:15.365119 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbed5a8c-7271-4749-b8b3-d1f8ba29bd8c admin admin] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:01:15.365408 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbed5a8c-7271-4749-b8b3-d1f8ba29bd8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:01:15.365678 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbed5a8c-7271-4749-b8b3-d1f8ba29bd8c admin admin] PATH_INFO: `/v3/users` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:01:15.366302 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbed5a8c-7271-4749-b8b3-d1f8ba29bd8c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:01:15.368573 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbed5a8c-7271-4749-b8b3-d1f8ba29bd8c admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:01:15.391693 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1141/2305] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Aug 30 16:01:15 2022] POST /identity/v3/users => generated 299 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 16:01:15.395825 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72692b4-758a-4dc0-81f5-4be446518823 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:01:15.406001 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72692b4-758a-4dc0-81f5-4be446518823 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:01:15.410680 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1621d22d-c3ff-43c5-a30a-eb8b98e32ba2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:01:15.411292 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1621d22d-c3ff-43c5-a30a-eb8b98e32ba2 admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:01:15.411568 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1621d22d-c3ff-43c5-a30a-eb8b98e32ba2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:01:15.411836 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1621d22d-c3ff-43c5-a30a-eb8b98e32ba2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:01:15.412489 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1621d22d-c3ff-43c5-a30a-eb8b98e32ba2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:01:15.414116 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1621d22d-c3ff-43c5-a30a-eb8b98e32ba2 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:01:15.424682 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1165/2306] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 30 16:01:15 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 16:01:15.428699 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbed5a8c-7271-4749-b8b3-d1f8ba29bd8c admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:01:15.438841 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbed5a8c-7271-4749-b8b3-d1f8ba29bd8c admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:01:15.444915 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d19ed3e-5bd8-4916-a5a0-ddb12dd15dc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:01:15.445564 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d19ed3e-5bd8-4916-a5a0-ddb12dd15dc8 admin admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:01:15.445823 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d19ed3e-5bd8-4916-a5a0-ddb12dd15dc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:01:15.446282 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d19ed3e-5bd8-4916-a5a0-ddb12dd15dc8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:01:15.447111 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d19ed3e-5bd8-4916-a5a0-ddb12dd15dc8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:01:15.448694 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d19ed3e-5bd8-4916-a5a0-ddb12dd15dc8 admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:01:15.461298 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1142/2307] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 30 16:01:15 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:01:15.466380 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1621d22d-c3ff-43c5-a30a-eb8b98e32ba2 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:01:15.476862 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1621d22d-c3ff-43c5-a30a-eb8b98e32ba2 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:01:15.481651 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3050f893-9c4a-4e6d-892d-89861290b842 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:01:15.482292 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3050f893-9c4a-4e6d-892d-89861290b842 admin admin] REQUEST_METHOD: `PUT` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:01:15.482569 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3050f893-9c4a-4e6d-892d-89861290b842 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:01:15.482840 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3050f893-9c4a-4e6d-892d-89861290b842 admin admin] PATH_INFO: `/v3/projects/06e3f7fc30ad481bbe4de9cf2afa9440/users/6883660efc634cc8bd4ebd24badec3bb/roles/df8f9f844b1b439181d75a07c1a93418` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:01:15.502888 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3050f893-9c4a-4e6d-892d-89861290b842 admin admin] RBAC: Authorizing `identity:create_grant(project_id=06e3f7fc30ad481bbe4de9cf2afa9440, user_id=6883660efc634cc8bd4ebd24badec3bb, role_id=df8f9f844b1b439181d75a07c1a93418)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:01:15.505597 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3050f893-9c4a-4e6d-892d-89861290b842 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:01:15.520245 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1166/2308] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 30 16:01:15 2022] PUT /identity/v3/projects/06e3f7fc30ad481bbe4de9cf2afa9440/users/6883660efc634cc8bd4ebd24badec3bb/roles/df8f9f844b1b439181d75a07c1a93418 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:01:15.532047 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fed7a6-d712-4a55-8117-9a6ce2b2aba2 None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:01:15.532468 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fed7a6-d712-4a55-8117-9a6ce2b2aba2 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:01:15.532904 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fed7a6-d712-4a55-8117-9a6ce2b2aba2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:01:15.563872 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-07fed7a6-d712-4a55-8117-9a6ce2b2aba2 None None] MFA Rules not processed for user `6883660efc634cc8bd4ebd24badec3bb`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:01:15.565275 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-07fed7a6-d712-4a55-8117-9a6ce2b2aba2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:01:15.575030 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:01:15.661269 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1143/2309] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 16:01:15 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:01:15.680251 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3050f893-9c4a-4e6d-892d-89861290b842 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:01:15.682098 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-3050f893-9c4a-4e6d-892d-89861290b842 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:01:15.742585 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3050f893-9c4a-4e6d-892d-89861290b842 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:01:15.750019 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2baf3e5-a678-4984-afca-d58ed12504de service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0100a04821804993866ea20f159663f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:01:15.750692 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2baf3e5-a678-4984-afca-d58ed12504de service cinder] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:01:15.750917 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2baf3e5-a678-4984-afca-d58ed12504de service cinder] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:01:15.751181 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2baf3e5-a678-4984-afca-d58ed12504de service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:01:15.760193 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2baf3e5-a678-4984-afca-d58ed12504de service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:01:15.761703 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2baf3e5-a678-4984-afca-d58ed12504de service cinder] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:01:15.770883 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1167/2310] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 16:01:15 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 16:01:24.024348 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fed7a6-d712-4a55-8117-9a6ce2b2aba2 None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:01:24.154198 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fed7a6-d712-4a55-8117-9a6ce2b2aba2 None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:01:24.159485 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0157bde3-e4d9-4f32-a19c-80f36e345562 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:01:24.160296 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0157bde3-e4d9-4f32-a19c-80f36e345562 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:01:24.160900 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0157bde3-e4d9-4f32-a19c-80f36e345562 admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:01:24.161647 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0157bde3-e4d9-4f32-a19c-80f36e345562 admin admin] PATH_INFO: `/v3/users/6883660efc634cc8bd4ebd24badec3bb` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:01:24.164461 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0157bde3-e4d9-4f32-a19c-80f36e345562 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6883660efc634cc8bd4ebd24badec3bb)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:01:24.167801 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0157bde3-e4d9-4f32-a19c-80f36e345562 admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:01:24.242964 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-0157bde3-e4d9-4f32-a19c-80f36e345562 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6883660efc634cc8bd4ebd24badec3bb'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:01:24.253252 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2baf3e5-a678-4984-afca-d58ed12504de service cinder] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:01:24.254548 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-0157bde3-e4d9-4f32-a19c-80f36e345562 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6883660efc634cc8bd4ebd24badec3bb'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:01:24.254836 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-c2baf3e5-a678-4984-afca-d58ed12504de service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:01:24.270016 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2baf3e5-a678-4984-afca-d58ed12504de service cinder] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:01:24.270240 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-0157bde3-e4d9-4f32-a19c-80f36e345562 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6883660efc634cc8bd4ebd24badec3bb'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:01:24.275085 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9479fd-d832-4b16-b860-b4aa6730c020 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:01:24.275862 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9479fd-d832-4b16-b860-b4aa6730c020 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:01:24.276168 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9479fd-d832-4b16-b860-b4aa6730c020 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:01:24.276767 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9479fd-d832-4b16-b860-b4aa6730c020 admin admin] PATH_INFO: `/v3/users/3028c6c8fbc443a1a1d4395961e173df` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:01:24.277956 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9479fd-d832-4b16-b860-b4aa6730c020 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3028c6c8fbc443a1a1d4395961e173df)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:01:24.278426 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-0157bde3-e4d9-4f32-a19c-80f36e345562 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6883660efc634cc8bd4ebd24badec3bb'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:01:24.279166 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-0157bde3-e4d9-4f32-a19c-80f36e345562 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6883660efc634cc8bd4ebd24badec3bb'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:01:24.280039 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9479fd-d832-4b16-b860-b4aa6730c020 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:01:24.289333 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1144/2311] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Aug 30 16:01:24 2022] DELETE /identity/v3/users/6883660efc634cc8bd4ebd24badec3bb => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:01:24.292298 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc57891-5a63-4643-b572-04125b0d17c1 None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:01:24.292814 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc57891-5a63-4643-b572-04125b0d17c1 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:01:24.293299 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc57891-5a63-4643-b572-04125b0d17c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:01:24.318827 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-dfc57891-5a63-4643-b572-04125b0d17c1 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:01:24.320294 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-dfc57891-5a63-4643-b572-04125b0d17c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:01:24.353760 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:01:24.355242 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-ba9479fd-d832-4b16-b860-b4aa6730c020 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3028c6c8fbc443a1a1d4395961e173df'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:01:24.369705 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-ba9479fd-d832-4b16-b860-b4aa6730c020 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3028c6c8fbc443a1a1d4395961e173df'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:01:24.378045 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-ba9479fd-d832-4b16-b860-b4aa6730c020 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3028c6c8fbc443a1a1d4395961e173df'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:01:24.378652 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-ba9479fd-d832-4b16-b860-b4aa6730c020 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3028c6c8fbc443a1a1d4395961e173df'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:01:24.385756 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-ba9479fd-d832-4b16-b860-b4aa6730c020 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3028c6c8fbc443a1a1d4395961e173df'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:01:24.389111 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1145/2312] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 16:01:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:01:24.392085 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1168/2313] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Aug 30 16:01:24 2022] DELETE /identity/v3/users/3028c6c8fbc443a1a1d4395961e173df => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:01:24.396544 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b08de84-2735-4fe4-92c0-69f9575f8958 None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:01:24.396894 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b08de84-2735-4fe4-92c0-69f9575f8958 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:01:24.397166 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b08de84-2735-4fe4-92c0-69f9575f8958 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:01:24.399750 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9479fd-d832-4b16-b860-b4aa6730c020 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:01:24.401143 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-ba9479fd-d832-4b16-b860-b4aa6730c020 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:01:24.410770 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:01:24.417031 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-2b08de84-2735-4fe4-92c0-69f9575f8958 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:01:24.418446 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-2b08de84-2735-4fe4-92c0-69f9575f8958 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:01:24.459898 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9479fd-d832-4b16-b860-b4aa6730c020 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:01:24.464137 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e811a24-7577-4605-b9c2-885c729f8973 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f4157105f3488f807b7475008ab722', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:01:24.464883 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e811a24-7577-4605-b9c2-885c729f8973 service neutron] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:01:24.465287 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e811a24-7577-4605-b9c2-885c729f8973 service neutron] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:01:24.465742 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e811a24-7577-4605-b9c2-885c729f8973 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:01:24.476544 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1146/2314] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 16:01:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 16:01:24.486302 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b08de84-2735-4fe4-92c0-69f9575f8958 None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:01:24.495949 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b08de84-2735-4fe4-92c0-69f9575f8958 None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:01:24.499499 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070d7ced-d017-4a88-99b0-9a1f3cfef787 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f4157105f3488f807b7475008ab722', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:01:24.500090 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070d7ced-d017-4a88-99b0-9a1f3cfef787 service neutron] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:01:24.500359 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070d7ced-d017-4a88-99b0-9a1f3cfef787 service neutron] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:01:24.500623 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070d7ced-d017-4a88-99b0-9a1f3cfef787 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:01:24.509776 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070d7ced-d017-4a88-99b0-9a1f3cfef787 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:01:24.511344 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070d7ced-d017-4a88-99b0-9a1f3cfef787 service neutron] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:01:24.521578 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1147/2315] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 16:01:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 16:01:24.532981 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e811a24-7577-4605-b9c2-885c729f8973 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:01:24.534558 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e811a24-7577-4605-b9c2-885c729f8973 service neutron] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:01:24.545300 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1169/2316] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 16:01:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 16:01:25.612582 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070d7ced-d017-4a88-99b0-9a1f3cfef787 service neutron] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:01:25.614467 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-070d7ced-d017-4a88-99b0-9a1f3cfef787 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:01:25.638077 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070d7ced-d017-4a88-99b0-9a1f3cfef787 service neutron] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:01:25.642868 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e811a24-7577-4605-b9c2-885c729f8973 service neutron] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:01:25.644025 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5143fa75-d580-44a3-97bd-1b419c277870 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:01:25.644447 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-9e811a24-7577-4605-b9c2-885c729f8973 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:01:25.644870 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5143fa75-d580-44a3-97bd-1b419c277870 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:01:25.645154 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5143fa75-d580-44a3-97bd-1b419c277870 admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:01:25.645460 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5143fa75-d580-44a3-97bd-1b419c277870 admin admin] PATH_INFO: `/v3/projects/def3b95b29d04e3680f087590bfcc8a4` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:01:25.646847 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5143fa75-d580-44a3-97bd-1b419c277870 admin admin] RBAC: Authorizing `identity:delete_project(project_id=def3b95b29d04e3680f087590bfcc8a4)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:01:25.649035 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5143fa75-d580-44a3-97bd-1b419c277870 admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:01:25.661356 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e811a24-7577-4605-b9c2-885c729f8973 service neutron] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:01:25.668686 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d7fe7c-5971-4d6a-9263-6639f57187e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:01:25.669671 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d7fe7c-5971-4d6a-9263-6639f57187e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:01:25.670128 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d7fe7c-5971-4d6a-9263-6639f57187e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:01:25.670406 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d7fe7c-5971-4d6a-9263-6639f57187e0 admin admin] PATH_INFO: `/v3/projects/06e3f7fc30ad481bbe4de9cf2afa9440` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:01:25.671603 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d7fe7c-5971-4d6a-9263-6639f57187e0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=06e3f7fc30ad481bbe4de9cf2afa9440)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:01:25.673891 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d7fe7c-5971-4d6a-9263-6639f57187e0 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:01:25.708262 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-5143fa75-d580-44a3-97bd-1b419c277870 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'def3b95b29d04e3680f087590bfcc8a4', 'request_id': 'req-5143fa75-d580-44a3-97bd-1b419c277870'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:01:25.715946 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-5143fa75-d580-44a3-97bd-1b419c277870 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'def3b95b29d04e3680f087590bfcc8a4', 'request_id': 'req-5143fa75-d580-44a3-97bd-1b419c277870'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:01:25.722839 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-5143fa75-d580-44a3-97bd-1b419c277870 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'def3b95b29d04e3680f087590bfcc8a4', 'request_id': 'req-5143fa75-d580-44a3-97bd-1b419c277870'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:01:25.728373 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-a8d7fe7c-5971-4d6a-9263-6639f57187e0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '06e3f7fc30ad481bbe4de9cf2afa9440', 'request_id': 'req-a8d7fe7c-5971-4d6a-9263-6639f57187e0'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:01:25.735627 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-5143fa75-d580-44a3-97bd-1b419c277870 admin admin] The token cache is being invalidate because project def3b95b29d04e3680f087590bfcc8a4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81512) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 16:01:25.735920 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-5143fa75-d580-44a3-97bd-1b419c277870 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:01:25.736583 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-5143fa75-d580-44a3-97bd-1b419c277870 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:01:25.736814 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-a8d7fe7c-5971-4d6a-9263-6639f57187e0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '06e3f7fc30ad481bbe4de9cf2afa9440', 'request_id': 'req-a8d7fe7c-5971-4d6a-9263-6639f57187e0'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:01:25.738803 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1148/2317] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Aug 30 16:01:25 2022] DELETE /identity/v3/projects/def3b95b29d04e3680f087590bfcc8a4 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:01:25.742770 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5143fa75-d580-44a3-97bd-1b419c277870 admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:01:25.744136 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-5143fa75-d580-44a3-97bd-1b419c277870 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:01:25.747063 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-a8d7fe7c-5971-4d6a-9263-6639f57187e0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '06e3f7fc30ad481bbe4de9cf2afa9440', 'request_id': 'req-a8d7fe7c-5971-4d6a-9263-6639f57187e0'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:01:25.758883 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-a8d7fe7c-5971-4d6a-9263-6639f57187e0 admin admin] The token cache is being invalidate because project 06e3f7fc30ad481bbe4de9cf2afa9440 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81511) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 16:01:25.759257 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-a8d7fe7c-5971-4d6a-9263-6639f57187e0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:01:25.760036 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-a8d7fe7c-5971-4d6a-9263-6639f57187e0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:01:25.761658 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1170/2318] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Aug 30 16:01:25 2022] DELETE /identity/v3/projects/06e3f7fc30ad481bbe4de9cf2afa9440 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 16:01:25.767890 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94711bc4-8446-40e7-bc6f-d2d0e5db3d11 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:01:25.768206 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94711bc4-8446-40e7-bc6f-d2d0e5db3d11 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:01:25.768496 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94711bc4-8446-40e7-bc6f-d2d0e5db3d11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:01:25.788803 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-94711bc4-8446-40e7-bc6f-d2d0e5db3d11 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:01:25.790063 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-94711bc4-8446-40e7-bc6f-d2d0e5db3d11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:01:25.790866 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:01:25.812493 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5143fa75-d580-44a3-97bd-1b419c277870 admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:01:25.832995 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:01:25.869688 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1171/2319] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 16:01:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 16:01:25.879641 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d5de46-2dca-4208-bab7-a216bf7d94f3 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:01:25.879920 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d5de46-2dca-4208-bab7-a216bf7d94f3 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:01:25.880189 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d5de46-2dca-4208-bab7-a216bf7d94f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:01:25.881099 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0455fc34-dd8c-43b9-8955-dd4283235689 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:01:25.881809 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0455fc34-dd8c-43b9-8955-dd4283235689 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:01:25.882090 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0455fc34-dd8c-43b9-8955-dd4283235689 admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:01:25.882361 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0455fc34-dd8c-43b9-8955-dd4283235689 admin admin] PATH_INFO: `/v3/users/c9ca2b72efda42498a45d5ed2129fcbe` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:01:25.883542 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0455fc34-dd8c-43b9-8955-dd4283235689 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c9ca2b72efda42498a45d5ed2129fcbe)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:01:25.885619 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0455fc34-dd8c-43b9-8955-dd4283235689 admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:01:25.901275 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-37d5de46-2dca-4208-bab7-a216bf7d94f3 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:01:25.902936 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-37d5de46-2dca-4208-bab7-a216bf7d94f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:01:25.913180 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1172/2320] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 16:01:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:01:25.918128 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d5de46-2dca-4208-bab7-a216bf7d94f3 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:01:25.929276 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d5de46-2dca-4208-bab7-a216bf7d94f3 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:01:25.938110 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2dedd5b-0086-4cfc-93d1-6afb494c6e2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:01:25.939194 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dedd5b-0086-4cfc-93d1-6afb494c6e2a admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:01:25.939477 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dedd5b-0086-4cfc-93d1-6afb494c6e2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:01:25.939748 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dedd5b-0086-4cfc-93d1-6afb494c6e2a admin admin] PATH_INFO: `/v3/domains` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:01:25.940448 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2dedd5b-0086-4cfc-93d1-6afb494c6e2a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:01:25.941053 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2dedd5b-0086-4cfc-93d1-6afb494c6e2a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:01:25.942814 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2dedd5b-0086-4cfc-93d1-6afb494c6e2a admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:01:25.950868 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-0455fc34-dd8c-43b9-8955-dd4283235689 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c9ca2b72efda42498a45d5ed2129fcbe'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:01:25.955423 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1173/2321] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Aug 30 16:01:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 16:01:25.960395 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2dedd5b-0086-4cfc-93d1-6afb494c6e2a admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:01:25.960671 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-0455fc34-dd8c-43b9-8955-dd4283235689 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c9ca2b72efda42498a45d5ed2129fcbe'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:01:25.973001 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2dedd5b-0086-4cfc-93d1-6afb494c6e2a admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:01:25.977978 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039c8207-c912-43c9-bafe-d39eefd1d602 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:01:25.978507 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-0455fc34-dd8c-43b9-8955-dd4283235689 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c9ca2b72efda42498a45d5ed2129fcbe'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:01:25.978657 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039c8207-c912-43c9-bafe-d39eefd1d602 admin admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:01:25.978846 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039c8207-c912-43c9-bafe-d39eefd1d602 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:01:25.979213 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039c8207-c912-43c9-bafe-d39eefd1d602 admin admin] PATH_INFO: `/v3/projects` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:01:25.979764 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039c8207-c912-43c9-bafe-d39eefd1d602 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:01:25.981772 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039c8207-c912-43c9-bafe-d39eefd1d602 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:01:25.986344 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-0455fc34-dd8c-43b9-8955-dd4283235689 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c9ca2b72efda42498a45d5ed2129fcbe'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:01:25.987071 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-0455fc34-dd8c-43b9-8955-dd4283235689 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c9ca2b72efda42498a45d5ed2129fcbe'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:01:25.995738 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1149/2322] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Aug 30 16:01:25 2022] DELETE /identity/v3/users/c9ca2b72efda42498a45d5ed2129fcbe => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 16:01:26.024442 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1174/2323] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Aug 30 16:01:25 2022] POST /identity/v3/projects => generated 373 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:01:26.025772 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0455fc34-dd8c-43b9-8955-dd4283235689 admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:01:26.027659 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-0455fc34-dd8c-43b9-8955-dd4283235689 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:01:26.111949 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0455fc34-dd8c-43b9-8955-dd4283235689 admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:01:26.124995 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:01:26.179920 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d3cd06-553c-4b00-9c81-7ca9a14b77b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:01:26.180555 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d3cd06-553c-4b00-9c81-7ca9a14b77b1 admin admin] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:01:26.180955 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d3cd06-553c-4b00-9c81-7ca9a14b77b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:01:26.181470 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d3cd06-553c-4b00-9c81-7ca9a14b77b1 admin admin] PATH_INFO: `/v3/users` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:01:26.182495 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d3cd06-553c-4b00-9c81-7ca9a14b77b1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:01:26.184556 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d3cd06-553c-4b00-9c81-7ca9a14b77b1 admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:01:26.216992 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039c8207-c912-43c9-bafe-d39eefd1d602 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:01:26.217384 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1150/2324] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Aug 30 16:01:26 2022] POST /identity/v3/users => generated 287 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:01:26.228516 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039c8207-c912-43c9-bafe-d39eefd1d602 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:01:26.234139 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cecd468-77ef-490f-a24f-412080624f60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:01:26.234772 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cecd468-77ef-490f-a24f-412080624f60 admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:01:26.235120 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cecd468-77ef-490f-a24f-412080624f60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:01:26.235393 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cecd468-77ef-490f-a24f-412080624f60 admin admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:01:26.236041 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cecd468-77ef-490f-a24f-412080624f60 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:01:26.238663 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cecd468-77ef-490f-a24f-412080624f60 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:01:26.251497 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1175/2325] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 30 16:01:26 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 16:01:26.255985 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d3cd06-553c-4b00-9c81-7ca9a14b77b1 admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:01:26.266259 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d3cd06-553c-4b00-9c81-7ca9a14b77b1 admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:01:26.271061 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e8d681-0d21-408c-8e10-91db1e7a76e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:01:26.271765 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e8d681-0d21-408c-8e10-91db1e7a76e6 admin admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:01:26.272039 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e8d681-0d21-408c-8e10-91db1e7a76e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:01:26.272309 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e8d681-0d21-408c-8e10-91db1e7a76e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:01:26.272976 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e8d681-0d21-408c-8e10-91db1e7a76e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:01:26.274635 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e8d681-0d21-408c-8e10-91db1e7a76e6 admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:01:26.289449 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1151/2326] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 30 16:01:26 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:01:26.290818 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cecd468-77ef-490f-a24f-412080624f60 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:01:26.301793 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cecd468-77ef-490f-a24f-412080624f60 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:01:26.308056 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ceab006-5a63-4e20-9b53-eeafb42efd7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:01:26.308517 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ceab006-5a63-4e20-9b53-eeafb42efd7b admin admin] REQUEST_METHOD: `PUT` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:01:26.308788 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ceab006-5a63-4e20-9b53-eeafb42efd7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:01:26.309051 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ceab006-5a63-4e20-9b53-eeafb42efd7b admin admin] PATH_INFO: `/v3/projects/340ea9b1270443f58c051cfb5699caa5/users/4f00011535aa4836b1cc410dd5d97beb/roles/df8f9f844b1b439181d75a07c1a93418` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:01:26.327587 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ceab006-5a63-4e20-9b53-eeafb42efd7b admin admin] RBAC: Authorizing `identity:create_grant(project_id=340ea9b1270443f58c051cfb5699caa5, user_id=4f00011535aa4836b1cc410dd5d97beb, role_id=df8f9f844b1b439181d75a07c1a93418)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:01:26.330828 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ceab006-5a63-4e20-9b53-eeafb42efd7b admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:01:26.348511 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1176/2327] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 30 16:01:26 2022] PUT /identity/v3/projects/340ea9b1270443f58c051cfb5699caa5/users/4f00011535aa4836b1cc410dd5d97beb/roles/df8f9f844b1b439181d75a07c1a93418 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:01:26.354872 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5ee6c2-e5db-49f1-89b6-5e7bcb6adb0f None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:01:26.355201 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5ee6c2-e5db-49f1-89b6-5e7bcb6adb0f None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:01:26.355518 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5ee6c2-e5db-49f1-89b6-5e7bcb6adb0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:01:26.391536 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-ed5ee6c2-e5db-49f1-89b6-5e7bcb6adb0f None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:01:26.392813 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-ed5ee6c2-e5db-49f1-89b6-5e7bcb6adb0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:01:26.421023 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ceab006-5a63-4e20-9b53-eeafb42efd7b admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:01:26.422963 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-1ceab006-5a63-4e20-9b53-eeafb42efd7b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:01:26.501169 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1152/2328] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 16:01:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:01:26.509358 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5ee6c2-e5db-49f1-89b6-5e7bcb6adb0f None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:01:26.511294 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-ed5ee6c2-e5db-49f1-89b6-5e7bcb6adb0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:01:26.540472 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ceab006-5a63-4e20-9b53-eeafb42efd7b admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:01:26.546743 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81154c5-3432-4851-9e06-97a79adbb292 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:01:26.547778 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81154c5-3432-4851-9e06-97a79adbb292 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:01:26.548129 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81154c5-3432-4851-9e06-97a79adbb292 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:01:26.548475 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81154c5-3432-4851-9e06-97a79adbb292 admin admin] PATH_INFO: `/v3/projects/f2a878ad1d794e6f89ad23bc44c4b7fa` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:01:26.550408 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81154c5-3432-4851-9e06-97a79adbb292 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f2a878ad1d794e6f89ad23bc44c4b7fa)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:01:26.552971 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81154c5-3432-4851-9e06-97a79adbb292 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:01:26.597882 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5ee6c2-e5db-49f1-89b6-5e7bcb6adb0f None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:01:26.603622 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d93b0c-d5da-4a79-bdff-b720967f5fa4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f4157105f3488f807b7475008ab722', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:01:26.604899 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d93b0c-d5da-4a79-bdff-b720967f5fa4 service neutron] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:01:26.605404 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d93b0c-d5da-4a79-bdff-b720967f5fa4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:01:26.605920 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d93b0c-d5da-4a79-bdff-b720967f5fa4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:01:26.619688 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d93b0c-d5da-4a79-bdff-b720967f5fa4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:01:26.622457 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d93b0c-d5da-4a79-bdff-b720967f5fa4 service neutron] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:01:26.638295 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-f81154c5-3432-4851-9e06-97a79adbb292 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f2a878ad1d794e6f89ad23bc44c4b7fa', 'request_id': 'req-f81154c5-3432-4851-9e06-97a79adbb292'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:01:26.640097 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1153/2329] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 16:01:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 16:01:26.654220 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-f81154c5-3432-4851-9e06-97a79adbb292 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f2a878ad1d794e6f89ad23bc44c4b7fa', 'request_id': 'req-f81154c5-3432-4851-9e06-97a79adbb292'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:01:26.663657 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-f81154c5-3432-4851-9e06-97a79adbb292 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f2a878ad1d794e6f89ad23bc44c4b7fa', 'request_id': 'req-f81154c5-3432-4851-9e06-97a79adbb292'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:01:26.668452 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=91,keystone:INSERT=3,keystone:DELETE=6 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:01:26.679889 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-f81154c5-3432-4851-9e06-97a79adbb292 admin admin] The token cache is being invalidate because project f2a878ad1d794e6f89ad23bc44c4b7fa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81511) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 16:01:26.680854 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-f81154c5-3432-4851-9e06-97a79adbb292 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:01:26.681748 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-f81154c5-3432-4851-9e06-97a79adbb292 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:01:26.684981 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1177/2330] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Aug 30 16:01:26 2022] DELETE /identity/v3/projects/f2a878ad1d794e6f89ad23bc44c4b7fa => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 16:01:26.689843 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5253307e-1291-49db-beb8-be1b8f8d79fc None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:01:26.690556 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5253307e-1291-49db-beb8-be1b8f8d79fc None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:01:26.690556 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5253307e-1291-49db-beb8-be1b8f8d79fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:01:26.720508 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-5253307e-1291-49db-beb8-be1b8f8d79fc None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:01:26.721792 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-5253307e-1291-49db-beb8-be1b8f8d79fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:01:26.759744 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:01:26.787540 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1154/2331] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 16:01:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:01:26.795481 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89597eb0-1ff5-42d0-9009-e63dc2f123ed None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:01:26.795759 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89597eb0-1ff5-42d0-9009-e63dc2f123ed None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:01:26.796029 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89597eb0-1ff5-42d0-9009-e63dc2f123ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:01:26.822109 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-89597eb0-1ff5-42d0-9009-e63dc2f123ed None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:01:26.823634 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-89597eb0-1ff5-42d0-9009-e63dc2f123ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:01:26.834678 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1178/2332] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 16:01:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:01:26.836166 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5253307e-1291-49db-beb8-be1b8f8d79fc None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:01:26.846366 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5253307e-1291-49db-beb8-be1b8f8d79fc None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:01:26.851072 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3513436-9c1c-4696-b483-d693c5f34a67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:01:26.851678 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3513436-9c1c-4696-b483-d693c5f34a67 admin admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:01:26.852123 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3513436-9c1c-4696-b483-d693c5f34a67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:01:26.852257 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3513436-9c1c-4696-b483-d693c5f34a67 admin admin] PATH_INFO: `/v3/domains` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:01:26.852856 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3513436-9c1c-4696-b483-d693c5f34a67 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81512) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:01:26.853157 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3513436-9c1c-4696-b483-d693c5f34a67 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:01:26.854796 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3513436-9c1c-4696-b483-d693c5f34a67 admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:01:26.869286 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1155/2333] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Aug 30 16:01:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 16:01:26.873681 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89597eb0-1ff5-42d0-9009-e63dc2f123ed None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:01:26.883323 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89597eb0-1ff5-42d0-9009-e63dc2f123ed None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:01:26.888243 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37c273f-e750-41c1-9efb-25c05597333e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:01:26.888846 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37c273f-e750-41c1-9efb-25c05597333e admin admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:01:26.889118 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37c273f-e750-41c1-9efb-25c05597333e admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:01:26.889416 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37c273f-e750-41c1-9efb-25c05597333e admin admin] PATH_INFO: `/v3/projects` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:01:26.890044 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37c273f-e750-41c1-9efb-25c05597333e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:01:26.891944 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37c273f-e750-41c1-9efb-25c05597333e admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:01:26.925332 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1179/2334] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Aug 30 16:01:26 2022] POST /identity/v3/projects => generated 383 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:01:26.926921 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3513436-9c1c-4696-b483-d693c5f34a67 admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:01:26.937277 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3513436-9c1c-4696-b483-d693c5f34a67 admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:01:26.998472 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d05445-16c7-4295-8354-b17e8e7a6506 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:01:26.999094 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d05445-16c7-4295-8354-b17e8e7a6506 admin admin] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:01:26.999543 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d05445-16c7-4295-8354-b17e8e7a6506 admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:01:26.999800 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d05445-16c7-4295-8354-b17e8e7a6506 admin admin] PATH_INFO: `/v3/users` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:01:27.000717 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d05445-16c7-4295-8354-b17e8e7a6506 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:01:27.002711 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d05445-16c7-4295-8354-b17e8e7a6506 admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:01:27.026510 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1156/2335] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Aug 30 16:01:26 2022] POST /identity/v3/users => generated 292 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:01:27.030664 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37c273f-e750-41c1-9efb-25c05597333e admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:01:27.047909 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37c273f-e750-41c1-9efb-25c05597333e admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:01:27.055504 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c9d391-b0be-49c6-8b1c-abaefe4fc2ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:01:27.056659 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c9d391-b0be-49c6-8b1c-abaefe4fc2ec admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:01:27.057200 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c9d391-b0be-49c6-8b1c-abaefe4fc2ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:01:27.057549 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c9d391-b0be-49c6-8b1c-abaefe4fc2ec admin admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:01:27.058408 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c9d391-b0be-49c6-8b1c-abaefe4fc2ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:01:27.069723 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c9d391-b0be-49c6-8b1c-abaefe4fc2ec admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:01:27.088499 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1180/2336] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 30 16:01:27 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:01:27.089868 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d05445-16c7-4295-8354-b17e8e7a6506 admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:01:27.105280 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d05445-16c7-4295-8354-b17e8e7a6506 admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:01:27.111889 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154aa20f-a258-44e3-acbe-62243579e567 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:01:27.112518 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154aa20f-a258-44e3-acbe-62243579e567 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:01:27.116682 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154aa20f-a258-44e3-acbe-62243579e567 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:01:27.118869 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc26a44-76c1-4ecb-83c3-8ec8134c9bd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:01:27.119790 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc26a44-76c1-4ecb-83c3-8ec8134c9bd7 admin admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:01:27.120277 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc26a44-76c1-4ecb-83c3-8ec8134c9bd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:01:27.121349 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc26a44-76c1-4ecb-83c3-8ec8134c9bd7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:01:27.122272 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc26a44-76c1-4ecb-83c3-8ec8134c9bd7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:01:27.124751 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc26a44-76c1-4ecb-83c3-8ec8134c9bd7 admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:01:27.138543 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1157/2337] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 30 16:01:27 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 16:01:27.142906 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc26a44-76c1-4ecb-83c3-8ec8134c9bd7 admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:01:27.150778 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-154aa20f-a258-44e3-acbe-62243579e567 None None] MFA Rules not processed for user `e9fdf5c5f263474bb41ab41bdac6f78f`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:01:27.152443 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-154aa20f-a258-44e3-acbe-62243579e567 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:01:27.156871 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc26a44-76c1-4ecb-83c3-8ec8134c9bd7 admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:01:27.164216 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e07230f-bece-4524-8236-b0d20b0fbb4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:01:27.165657 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e07230f-bece-4524-8236-b0d20b0fbb4c admin admin] REQUEST_METHOD: `PUT` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:01:27.166040 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e07230f-bece-4524-8236-b0d20b0fbb4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:01:27.166558 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e07230f-bece-4524-8236-b0d20b0fbb4c admin admin] PATH_INFO: `/v3/projects/bbf5ce43649b4c7fb13db31293cd7b6c/users/7736a850a7cb4c22bb7ebcf020a38b16/roles/df8f9f844b1b439181d75a07c1a93418` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:01:27.189011 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e07230f-bece-4524-8236-b0d20b0fbb4c admin admin] RBAC: Authorizing `identity:create_grant(project_id=bbf5ce43649b4c7fb13db31293cd7b6c, user_id=7736a850a7cb4c22bb7ebcf020a38b16, role_id=df8f9f844b1b439181d75a07c1a93418)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:01:27.191964 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e07230f-bece-4524-8236-b0d20b0fbb4c admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:01:27.211993 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1158/2338] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 30 16:01:27 2022] PUT /identity/v3/projects/bbf5ce43649b4c7fb13db31293cd7b6c/users/7736a850a7cb4c22bb7ebcf020a38b16/roles/df8f9f844b1b439181d75a07c1a93418 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:01:27.223707 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88af01ed-4c8e-4085-a653-dea63f3254f5 None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:01:27.223976 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88af01ed-4c8e-4085-a653-dea63f3254f5 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:01:27.224246 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88af01ed-4c8e-4085-a653-dea63f3254f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:01:27.225978 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1181/2339] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 16:01:27 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:01:27.267489 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-88af01ed-4c8e-4085-a653-dea63f3254f5 None None] MFA Rules not processed for user `7736a850a7cb4c22bb7ebcf020a38b16`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:01:27.268791 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-88af01ed-4c8e-4085-a653-dea63f3254f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:01:27.352224 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1159/2340] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 16:01:27 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 16:01:27.357291 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154aa20f-a258-44e3-acbe-62243579e567 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:01:27.374117 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154aa20f-a258-44e3-acbe-62243579e567 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:01:27.451870 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d79e13-19a0-47ab-9026-c4f2e069d1b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:01:27.452890 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d79e13-19a0-47ab-9026-c4f2e069d1b1 admin admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:01:27.453424 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d79e13-19a0-47ab-9026-c4f2e069d1b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:01:27.453873 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d79e13-19a0-47ab-9026-c4f2e069d1b1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:01:27.454527 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d79e13-19a0-47ab-9026-c4f2e069d1b1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:01:27.457251 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d79e13-19a0-47ab-9026-c4f2e069d1b1 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:01:27.499642 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1182/2341] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Aug 30 16:01:27 2022] POST /identity/v3/projects => generated 383 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:01:27.500850 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88af01ed-4c8e-4085-a653-dea63f3254f5 None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:01:27.514171 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88af01ed-4c8e-4085-a653-dea63f3254f5 None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:01:27.519538 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:01:27.583806 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746a9f26-94ce-409a-8e6b-1b05fc08b65f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:01:27.584451 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746a9f26-94ce-409a-8e6b-1b05fc08b65f admin admin] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:01:27.584731 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746a9f26-94ce-409a-8e6b-1b05fc08b65f admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:01:27.585005 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746a9f26-94ce-409a-8e6b-1b05fc08b65f admin admin] PATH_INFO: `/v3/users` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:01:27.585694 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746a9f26-94ce-409a-8e6b-1b05fc08b65f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:01:27.587642 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746a9f26-94ce-409a-8e6b-1b05fc08b65f admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:01:27.611889 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1160/2342] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Aug 30 16:01:27 2022] POST /identity/v3/users => generated 298 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:01:27.616437 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d79e13-19a0-47ab-9026-c4f2e069d1b1 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:01:27.633189 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d79e13-19a0-47ab-9026-c4f2e069d1b1 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:01:27.642999 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06143035-05d2-4f03-9d55-4fd5b7755bd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:01:27.643637 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06143035-05d2-4f03-9d55-4fd5b7755bd1 admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:01:27.643918 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06143035-05d2-4f03-9d55-4fd5b7755bd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:01:27.644192 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06143035-05d2-4f03-9d55-4fd5b7755bd1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:01:27.644916 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06143035-05d2-4f03-9d55-4fd5b7755bd1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:01:27.646825 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06143035-05d2-4f03-9d55-4fd5b7755bd1 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:01:27.659087 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1183/2343] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 30 16:01:27 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:01:27.663663 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746a9f26-94ce-409a-8e6b-1b05fc08b65f admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:01:27.675187 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746a9f26-94ce-409a-8e6b-1b05fc08b65f admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:01:27.680359 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdacc28-45d8-4fdb-97ee-52411b48f88b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:01:27.681040 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdacc28-45d8-4fdb-97ee-52411b48f88b admin admin] REQUEST_METHOD: `PUT` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:01:27.681349 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdacc28-45d8-4fdb-97ee-52411b48f88b admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:01:27.681616 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdacc28-45d8-4fdb-97ee-52411b48f88b admin admin] PATH_INFO: `/v3/projects/93ee0f765b4145d5b955569bc372ad15/users/e7e8f73382fd4adc9e873c33ef5c4315/roles/f265386950a641ac8413b53eda4ae34d` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:01:27.700836 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfdacc28-45d8-4fdb-97ee-52411b48f88b admin admin] RBAC: Authorizing `identity:create_grant(project_id=93ee0f765b4145d5b955569bc372ad15, user_id=e7e8f73382fd4adc9e873c33ef5c4315, role_id=f265386950a641ac8413b53eda4ae34d)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:01:27.703637 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfdacc28-45d8-4fdb-97ee-52411b48f88b admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:01:27.721100 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1161/2344] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 30 16:01:27 2022] PUT /identity/v3/projects/93ee0f765b4145d5b955569bc372ad15/users/e7e8f73382fd4adc9e873c33ef5c4315/roles/f265386950a641ac8413b53eda4ae34d => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:01:27.725233 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06143035-05d2-4f03-9d55-4fd5b7755bd1 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:01:27.735761 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06143035-05d2-4f03-9d55-4fd5b7755bd1 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:01:27.769528 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:DELETE=1,keystone:SELECT=97,keystone:INSERT=2 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:01:27.804935 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d044cb-8405-4ae4-b31e-ac5c9d5417e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:01:27.805249 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d044cb-8405-4ae4-b31e-ac5c9d5417e2 admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:01:27.805460 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d044cb-8405-4ae4-b31e-ac5c9d5417e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:01:27.805728 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d044cb-8405-4ae4-b31e-ac5c9d5417e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:01:27.806373 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d044cb-8405-4ae4-b31e-ac5c9d5417e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:01:27.808043 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d044cb-8405-4ae4-b31e-ac5c9d5417e2 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:01:27.818800 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1184/2345] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 30 16:01:27 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 16:01:27.822656 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdacc28-45d8-4fdb-97ee-52411b48f88b admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:01:27.834020 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdacc28-45d8-4fdb-97ee-52411b48f88b admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:01:27.841396 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f767eb-4d19-49f6-bc4a-c46111ed3dc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:01:27.842433 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f767eb-4d19-49f6-bc4a-c46111ed3dc9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:01:27.842846 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f767eb-4d19-49f6-bc4a-c46111ed3dc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:01:27.843251 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f767eb-4d19-49f6-bc4a-c46111ed3dc9 admin admin] PATH_INFO: `/v3/projects/93ee0f765b4145d5b955569bc372ad15/users/e7e8f73382fd4adc9e873c33ef5c4315/roles/f265386950a641ac8413b53eda4ae34d` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:01:27.845407 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f767eb-4d19-49f6-bc4a-c46111ed3dc9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=93ee0f765b4145d5b955569bc372ad15, user_id=e7e8f73382fd4adc9e873c33ef5c4315, role_id=f265386950a641ac8413b53eda4ae34d)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:01:27.860824 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f767eb-4d19-49f6-bc4a-c46111ed3dc9 admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:01:27.885262 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1162/2346] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 30 16:01:27 2022] PUT /identity/v3/projects/93ee0f765b4145d5b955569bc372ad15/users/e7e8f73382fd4adc9e873c33ef5c4315/roles/f265386950a641ac8413b53eda4ae34d => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 16:01:27.885865 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451e7a3a-ada6-454c-b724-bd871e6d1cf6 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:01:27.886217 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451e7a3a-ada6-454c-b724-bd871e6d1cf6 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:01:27.886466 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451e7a3a-ada6-454c-b724-bd871e6d1cf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:01:27.935344 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-451e7a3a-ada6-454c-b724-bd871e6d1cf6 None None] MFA Rules not processed for user `e7e8f73382fd4adc9e873c33ef5c4315`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:01:27.937005 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-451e7a3a-ada6-454c-b724-bd871e6d1cf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:01:28.030024 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1185/2347] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 16:01:27 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:01:28.047090 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f767eb-4d19-49f6-bc4a-c46111ed3dc9 admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:01:28.049169 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-a7f767eb-4d19-49f6-bc4a-c46111ed3dc9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:01:28.127230 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f767eb-4d19-49f6-bc4a-c46111ed3dc9 admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:01:28.137992 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c09eb91-09d2-424a-af87-f346d8e6ff91 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0100a04821804993866ea20f159663f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:01:28.138646 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c09eb91-09d2-424a-af87-f346d8e6ff91 service cinder] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:01:28.138907 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c09eb91-09d2-424a-af87-f346d8e6ff91 service cinder] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:01:28.140383 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c09eb91-09d2-424a-af87-f346d8e6ff91 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:01:28.155716 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c09eb91-09d2-424a-af87-f346d8e6ff91 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:01:28.157547 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c09eb91-09d2-424a-af87-f346d8e6ff91 service cinder] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:01:28.171173 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1163/2348] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 16:01:28 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:01:28.289825 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451e7a3a-ada6-454c-b724-bd871e6d1cf6 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:01:28.301374 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451e7a3a-ada6-454c-b724-bd871e6d1cf6 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:01:28.306840 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6204e19b-8d66-402e-bd7c-fb740c35c6f5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0100a04821804993866ea20f159663f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:01:28.307585 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6204e19b-8d66-402e-bd7c-fb740c35c6f5 service cinder] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:01:28.307899 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6204e19b-8d66-402e-bd7c-fb740c35c6f5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:01:28.308326 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6204e19b-8d66-402e-bd7c-fb740c35c6f5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:01:28.444931 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6204e19b-8d66-402e-bd7c-fb740c35c6f5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:01:28.446602 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6204e19b-8d66-402e-bd7c-fb740c35c6f5 service cinder] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:01:28.457425 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1186/2349] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 16:01:28 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:01:28.949610 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c09eb91-09d2-424a-af87-f346d8e6ff91 service cinder] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:01:29.036524 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c09eb91-09d2-424a-af87-f346d8e6ff91 service cinder] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:01:29.046513 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571cdcf1-9405-48de-b24f-c2d4b840f13e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:01:29.047458 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571cdcf1-9405-48de-b24f-c2d4b840f13e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:01:29.048266 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571cdcf1-9405-48de-b24f-c2d4b840f13e admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:01:29.048626 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571cdcf1-9405-48de-b24f-c2d4b840f13e admin admin] PATH_INFO: `/v3/users/7736a850a7cb4c22bb7ebcf020a38b16` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:01:29.049987 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571cdcf1-9405-48de-b24f-c2d4b840f13e admin admin] RBAC: Authorizing `identity:delete_user(user_id=7736a850a7cb4c22bb7ebcf020a38b16)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:01:29.052125 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571cdcf1-9405-48de-b24f-c2d4b840f13e admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:01:29.057899 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:01:29.124784 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-571cdcf1-9405-48de-b24f-c2d4b840f13e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7736a850a7cb4c22bb7ebcf020a38b16'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:01:29.138763 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-571cdcf1-9405-48de-b24f-c2d4b840f13e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7736a850a7cb4c22bb7ebcf020a38b16'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:01:29.156719 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-571cdcf1-9405-48de-b24f-c2d4b840f13e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7736a850a7cb4c22bb7ebcf020a38b16'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:01:29.166904 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-571cdcf1-9405-48de-b24f-c2d4b840f13e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7736a850a7cb4c22bb7ebcf020a38b16'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:01:29.167768 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-571cdcf1-9405-48de-b24f-c2d4b840f13e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7736a850a7cb4c22bb7ebcf020a38b16'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:01:29.174588 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1164/2350] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Aug 30 16:01:28 2022] DELETE /identity/v3/users/7736a850a7cb4c22bb7ebcf020a38b16 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:01:29.178971 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969ef9ff-1a95-4bfe-a244-d3ff850cab7d None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:01:29.179484 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969ef9ff-1a95-4bfe-a244-d3ff850cab7d None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:01:29.179709 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969ef9ff-1a95-4bfe-a244-d3ff850cab7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:01:29.204995 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-969ef9ff-1a95-4bfe-a244-d3ff850cab7d None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:01:29.207533 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-969ef9ff-1a95-4bfe-a244-d3ff850cab7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:01:29.265252 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:01:29.279737 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1187/2351] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 16:01:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 16:01:29.301749 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571cdcf1-9405-48de-b24f-c2d4b840f13e admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:01:29.303443 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-571cdcf1-9405-48de-b24f-c2d4b840f13e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:01:29.408021 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571cdcf1-9405-48de-b24f-c2d4b840f13e admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:01:29.412992 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad238d3a-43a9-437f-b259-6874254c2ba9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f4157105f3488f807b7475008ab722', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:01:29.414050 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad238d3a-43a9-437f-b259-6874254c2ba9 service neutron] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:01:29.414406 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad238d3a-43a9-437f-b259-6874254c2ba9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:01:29.414750 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad238d3a-43a9-437f-b259-6874254c2ba9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:01:29.442570 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad238d3a-43a9-437f-b259-6874254c2ba9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:01:29.445713 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad238d3a-43a9-437f-b259-6874254c2ba9 service neutron] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:01:29.468659 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1165/2352] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 16:01:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:01:29.998359 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969ef9ff-1a95-4bfe-a244-d3ff850cab7d None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:01:30.000271 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-969ef9ff-1a95-4bfe-a244-d3ff850cab7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:01:30.030795 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969ef9ff-1a95-4bfe-a244-d3ff850cab7d None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:01:30.035639 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3059a2c5-5750-426f-8fbc-14446d4bdc35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:01:30.036449 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3059a2c5-5750-426f-8fbc-14446d4bdc35 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:01:30.036733 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3059a2c5-5750-426f-8fbc-14446d4bdc35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:01:30.037624 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3059a2c5-5750-426f-8fbc-14446d4bdc35 admin admin] PATH_INFO: `/v3/projects/bbf5ce43649b4c7fb13db31293cd7b6c` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:01:30.038406 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3059a2c5-5750-426f-8fbc-14446d4bdc35 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bbf5ce43649b4c7fb13db31293cd7b6c)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:01:30.040807 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3059a2c5-5750-426f-8fbc-14446d4bdc35 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:01:30.115955 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-3059a2c5-5750-426f-8fbc-14446d4bdc35 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bbf5ce43649b4c7fb13db31293cd7b6c', 'request_id': 'req-3059a2c5-5750-426f-8fbc-14446d4bdc35'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:01:30.128834 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-3059a2c5-5750-426f-8fbc-14446d4bdc35 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bbf5ce43649b4c7fb13db31293cd7b6c', 'request_id': 'req-3059a2c5-5750-426f-8fbc-14446d4bdc35'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:01:30.138006 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-3059a2c5-5750-426f-8fbc-14446d4bdc35 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bbf5ce43649b4c7fb13db31293cd7b6c', 'request_id': 'req-3059a2c5-5750-426f-8fbc-14446d4bdc35'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:01:30.151152 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-3059a2c5-5750-426f-8fbc-14446d4bdc35 admin admin] The token cache is being invalidate because project bbf5ce43649b4c7fb13db31293cd7b6c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81511) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 16:01:30.151478 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-3059a2c5-5750-426f-8fbc-14446d4bdc35 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:01:30.152172 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-3059a2c5-5750-426f-8fbc-14446d4bdc35 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:01:30.161973 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1188/2353] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Aug 30 16:01:29 2022] DELETE /identity/v3/projects/bbf5ce43649b4c7fb13db31293cd7b6c => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:01:30.162628 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad238d3a-43a9-437f-b259-6874254c2ba9 service neutron] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:01:30.184153 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad238d3a-43a9-437f-b259-6874254c2ba9 service neutron] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:01:30.263366 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304d2e46-28b5-49ca-9f07-34c48f64d192 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:01:30.264443 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304d2e46-28b5-49ca-9f07-34c48f64d192 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:01:30.265670 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304d2e46-28b5-49ca-9f07-34c48f64d192 admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:01:30.267041 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304d2e46-28b5-49ca-9f07-34c48f64d192 admin admin] PATH_INFO: `/v3/users/e7e8f73382fd4adc9e873c33ef5c4315` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:01:30.269310 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304d2e46-28b5-49ca-9f07-34c48f64d192 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e7e8f73382fd4adc9e873c33ef5c4315)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:01:30.271888 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304d2e46-28b5-49ca-9f07-34c48f64d192 admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:01:30.316423 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:01:30.343826 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-304d2e46-28b5-49ca-9f07-34c48f64d192 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e7e8f73382fd4adc9e873c33ef5c4315'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:01:30.353017 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-304d2e46-28b5-49ca-9f07-34c48f64d192 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e7e8f73382fd4adc9e873c33ef5c4315'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:01:30.369842 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-304d2e46-28b5-49ca-9f07-34c48f64d192 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e7e8f73382fd4adc9e873c33ef5c4315'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:01:30.377334 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-304d2e46-28b5-49ca-9f07-34c48f64d192 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e7e8f73382fd4adc9e873c33ef5c4315'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:01:30.378125 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-304d2e46-28b5-49ca-9f07-34c48f64d192 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e7e8f73382fd4adc9e873c33ef5c4315'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:01:30.398460 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1166/2354] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Aug 30 16:01:30 2022] DELETE /identity/v3/users/e7e8f73382fd4adc9e873c33ef5c4315 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 16:01:30.735279 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3059a2c5-5750-426f-8fbc-14446d4bdc35 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:01:30.736834 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-3059a2c5-5750-426f-8fbc-14446d4bdc35 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:01:30.753538 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3059a2c5-5750-426f-8fbc-14446d4bdc35 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:01:30.817976 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fce4360-0805-4289-a994-421727d8d93c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:01:30.819275 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fce4360-0805-4289-a994-421727d8d93c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:01:30.819617 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fce4360-0805-4289-a994-421727d8d93c admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:01:30.820309 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fce4360-0805-4289-a994-421727d8d93c admin admin] PATH_INFO: `/v3/projects/93ee0f765b4145d5b955569bc372ad15` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:01:30.821706 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fce4360-0805-4289-a994-421727d8d93c admin admin] RBAC: Authorizing `identity:delete_project(project_id=93ee0f765b4145d5b955569bc372ad15)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:01:30.824198 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fce4360-0805-4289-a994-421727d8d93c admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:01:30.879272 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-9fce4360-0805-4289-a994-421727d8d93c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '93ee0f765b4145d5b955569bc372ad15', 'request_id': 'req-9fce4360-0805-4289-a994-421727d8d93c'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:01:30.890167 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-9fce4360-0805-4289-a994-421727d8d93c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '93ee0f765b4145d5b955569bc372ad15', 'request_id': 'req-9fce4360-0805-4289-a994-421727d8d93c'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:01:30.897922 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-9fce4360-0805-4289-a994-421727d8d93c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '93ee0f765b4145d5b955569bc372ad15', 'request_id': 'req-9fce4360-0805-4289-a994-421727d8d93c'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:01:30.912956 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-9fce4360-0805-4289-a994-421727d8d93c admin admin] The token cache is being invalidate because project 93ee0f765b4145d5b955569bc372ad15 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81511) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 16:01:30.913314 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-9fce4360-0805-4289-a994-421727d8d93c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:01:30.914011 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-9fce4360-0805-4289-a994-421727d8d93c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:01:30.915535 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1189/2355] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Aug 30 16:01:30 2022] DELETE /identity/v3/projects/93ee0f765b4145d5b955569bc372ad15 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:01:30.921006 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139227ec-d1c5-4d4d-a0fb-a6d3fe13098e None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:01:30.921425 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139227ec-d1c5-4d4d-a0fb-a6d3fe13098e None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:01:30.921708 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139227ec-d1c5-4d4d-a0fb-a6d3fe13098e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:01:30.942364 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-139227ec-d1c5-4d4d-a0fb-a6d3fe13098e None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:01:30.943680 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-139227ec-d1c5-4d4d-a0fb-a6d3fe13098e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:01:31.029380 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1167/2356] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 16:01:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:01:31.044770 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f594bc-8f56-433f-b204-07f6203a5a5b None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:01:31.045140 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f594bc-8f56-433f-b204-07f6203a5a5b None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:01:31.045497 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f594bc-8f56-433f-b204-07f6203a5a5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:01:31.071231 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:01:31.089743 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-20f594bc-8f56-433f-b204-07f6203a5a5b None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:01:31.091832 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-20f594bc-8f56-433f-b204-07f6203a5a5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:01:31.112636 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1190/2357] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 16:01:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:01:31.117073 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139227ec-d1c5-4d4d-a0fb-a6d3fe13098e None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:01:31.132050 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139227ec-d1c5-4d4d-a0fb-a6d3fe13098e None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:01:31.148080 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cf8594-d954-435f-a1d8-8cdae1d23c34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:01:31.149110 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cf8594-d954-435f-a1d8-8cdae1d23c34 admin admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:01:31.149437 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cf8594-d954-435f-a1d8-8cdae1d23c34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:01:31.149712 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cf8594-d954-435f-a1d8-8cdae1d23c34 admin admin] PATH_INFO: `/v3/domains` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:01:31.150665 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74cf8594-d954-435f-a1d8-8cdae1d23c34 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81512) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:01:31.150983 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74cf8594-d954-435f-a1d8-8cdae1d23c34 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:01:31.153720 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74cf8594-d954-435f-a1d8-8cdae1d23c34 admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:01:31.173262 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1168/2358] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Aug 30 16:01:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:01:31.177356 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f594bc-8f56-433f-b204-07f6203a5a5b None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:01:31.191234 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f594bc-8f56-433f-b204-07f6203a5a5b None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:01:31.197583 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eea32c2-82fa-4eab-ac32-c3854e0f01f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:01:31.198288 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eea32c2-82fa-4eab-ac32-c3854e0f01f0 admin admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:01:31.198657 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eea32c2-82fa-4eab-ac32-c3854e0f01f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:01:31.199096 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eea32c2-82fa-4eab-ac32-c3854e0f01f0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:01:31.200311 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eea32c2-82fa-4eab-ac32-c3854e0f01f0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:01:31.202792 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eea32c2-82fa-4eab-ac32-c3854e0f01f0 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:01:31.248974 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1191/2359] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Aug 30 16:01:31 2022] POST /identity/v3/projects => generated 377 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:01:31.253057 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cf8594-d954-435f-a1d8-8cdae1d23c34 admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:01:31.264000 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cf8594-d954-435f-a1d8-8cdae1d23c34 admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:01:31.327869 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda89023-2a1b-402d-8c50-2ed0d7ed1594 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:01:31.328422 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda89023-2a1b-402d-8c50-2ed0d7ed1594 admin admin] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:01:31.328716 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda89023-2a1b-402d-8c50-2ed0d7ed1594 admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:01:31.329021 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda89023-2a1b-402d-8c50-2ed0d7ed1594 admin admin] PATH_INFO: `/v3/users` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:01:31.329730 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda89023-2a1b-402d-8c50-2ed0d7ed1594 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:01:31.331838 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda89023-2a1b-402d-8c50-2ed0d7ed1594 admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:01:31.356749 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1169/2360] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Aug 30 16:01:31 2022] POST /identity/v3/users => generated 289 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:01:31.360680 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eea32c2-82fa-4eab-ac32-c3854e0f01f0 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:01:31.374040 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eea32c2-82fa-4eab-ac32-c3854e0f01f0 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:01:31.379721 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb92d6e3-3efe-4070-901e-f476576af40d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:01:31.380504 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb92d6e3-3efe-4070-901e-f476576af40d admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:01:31.380855 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb92d6e3-3efe-4070-901e-f476576af40d admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:01:31.384668 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb92d6e3-3efe-4070-901e-f476576af40d admin admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:01:31.385480 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb92d6e3-3efe-4070-901e-f476576af40d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:01:31.387251 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb92d6e3-3efe-4070-901e-f476576af40d admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:01:31.411975 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1192/2361] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 30 16:01:31 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 16:01:31.413945 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda89023-2a1b-402d-8c50-2ed0d7ed1594 admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:01:31.434447 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda89023-2a1b-402d-8c50-2ed0d7ed1594 admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:01:31.444473 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabf3652-7e52-4b40-b1d3-beb630213a83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:01:31.445887 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabf3652-7e52-4b40-b1d3-beb630213a83 admin admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:01:31.446811 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabf3652-7e52-4b40-b1d3-beb630213a83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:01:31.447097 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabf3652-7e52-4b40-b1d3-beb630213a83 admin admin] PATH_INFO: `/v3/roles` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:01:31.447764 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabf3652-7e52-4b40-b1d3-beb630213a83 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:01:31.449425 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabf3652-7e52-4b40-b1d3-beb630213a83 admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:01:31.485102 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1170/2362] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 30 16:01:31 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:01:31.488696 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb92d6e3-3efe-4070-901e-f476576af40d admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:01:31.510674 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb92d6e3-3efe-4070-901e-f476576af40d admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:01:31.522150 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4117a8c-5077-4ed4-9f68-f1db4830312a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:01:31.529068 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4117a8c-5077-4ed4-9f68-f1db4830312a admin admin] REQUEST_METHOD: `PUT` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:01:31.529422 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4117a8c-5077-4ed4-9f68-f1db4830312a admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:01:31.532246 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4117a8c-5077-4ed4-9f68-f1db4830312a admin admin] PATH_INFO: `/v3/projects/1622471368b54226a194dc3ea18db4c2/users/cb359499a0944dc583a225beb80e0989/roles/df8f9f844b1b439181d75a07c1a93418` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:01:31.573573 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4117a8c-5077-4ed4-9f68-f1db4830312a admin admin] RBAC: Authorizing `identity:create_grant(project_id=1622471368b54226a194dc3ea18db4c2, user_id=cb359499a0944dc583a225beb80e0989, role_id=df8f9f844b1b439181d75a07c1a93418)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:01:31.576171 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4117a8c-5077-4ed4-9f68-f1db4830312a admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:01:31.600702 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1193/2363] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 30 16:01:31 2022] PUT /identity/v3/projects/1622471368b54226a194dc3ea18db4c2/users/cb359499a0944dc583a225beb80e0989/roles/df8f9f844b1b439181d75a07c1a93418 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:01:31.612965 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb10ca21-8834-4203-a1a9-0b19ba82034c None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:01:31.613258 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb10ca21-8834-4203-a1a9-0b19ba82034c None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:01:31.613537 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb10ca21-8834-4203-a1a9-0b19ba82034c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:01:31.668355 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:01:31.681276 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-fb10ca21-8834-4203-a1a9-0b19ba82034c None None] MFA Rules not processed for user `cb359499a0944dc583a225beb80e0989`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:01:31.685373 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-fb10ca21-8834-4203-a1a9-0b19ba82034c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:01:31.833936 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1171/2364] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 16:01:31 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:01:31.839484 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4117a8c-5077-4ed4-9f68-f1db4830312a admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:01:31.923400 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4117a8c-5077-4ed4-9f68-f1db4830312a admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:01:31.929423 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5143c99-683e-43a6-b959-41639981ed2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:01:31.930322 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5143c99-683e-43a6-b959-41639981ed2b admin admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:01:31.930322 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5143c99-683e-43a6-b959-41639981ed2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:01:31.931211 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5143c99-683e-43a6-b959-41639981ed2b admin admin] PATH_INFO: `/v3/projects` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:01:31.931211 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5143c99-683e-43a6-b959-41639981ed2b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:01:31.933150 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5143c99-683e-43a6-b959-41639981ed2b admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:01:31.964359 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1194/2365] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Aug 30 16:01:31 2022] POST /identity/v3/projects => generated 375 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:01:31.969180 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb10ca21-8834-4203-a1a9-0b19ba82034c None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:01:31.981134 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb10ca21-8834-4203-a1a9-0b19ba82034c None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:01:32.051603 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b4811e-7fd3-4364-903e-691c690821e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:01:32.052228 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b4811e-7fd3-4364-903e-691c690821e4 admin admin] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:01:32.053040 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b4811e-7fd3-4364-903e-691c690821e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:01:32.053424 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b4811e-7fd3-4364-903e-691c690821e4 admin admin] PATH_INFO: `/v3/users` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:01:32.053993 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b4811e-7fd3-4364-903e-691c690821e4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:01:32.056049 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b4811e-7fd3-4364-903e-691c690821e4 admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:01:32.084489 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1172/2366] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Aug 30 16:01:31 2022] POST /identity/v3/users => generated 288 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:01:32.088413 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5143c99-683e-43a6-b959-41639981ed2b admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:01:32.100384 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5143c99-683e-43a6-b959-41639981ed2b admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:01:32.105737 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40a5bdb-1f6d-4ca7-b7bf-71b76a1e032d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:01:32.106503 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40a5bdb-1f6d-4ca7-b7bf-71b76a1e032d admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:01:32.106784 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40a5bdb-1f6d-4ca7-b7bf-71b76a1e032d admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:01:32.107049 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40a5bdb-1f6d-4ca7-b7bf-71b76a1e032d admin admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:01:32.107698 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40a5bdb-1f6d-4ca7-b7bf-71b76a1e032d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:01:32.109344 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40a5bdb-1f6d-4ca7-b7bf-71b76a1e032d admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:01:32.125938 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1195/2367] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 30 16:01:32 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 16:01:32.130535 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b4811e-7fd3-4364-903e-691c690821e4 admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:01:32.142610 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b4811e-7fd3-4364-903e-691c690821e4 admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:01:32.149121 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c682bf1a-2567-42e0-8ea2-4c8b15077e3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:01:32.149759 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c682bf1a-2567-42e0-8ea2-4c8b15077e3c admin admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:01:32.150107 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c682bf1a-2567-42e0-8ea2-4c8b15077e3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:01:32.150435 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c682bf1a-2567-42e0-8ea2-4c8b15077e3c admin admin] PATH_INFO: `/v3/roles` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:01:32.151169 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c682bf1a-2567-42e0-8ea2-4c8b15077e3c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:01:32.153635 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c682bf1a-2567-42e0-8ea2-4c8b15077e3c admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:01:32.172040 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40a5bdb-1f6d-4ca7-b7bf-71b76a1e032d admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:01:32.178835 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1173/2368] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 30 16:01:32 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:01:32.184579 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40a5bdb-1f6d-4ca7-b7bf-71b76a1e032d admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:01:32.191490 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825fd6ae-092f-425f-bc9d-dc217f91f4b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:01:32.192308 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825fd6ae-092f-425f-bc9d-dc217f91f4b8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:01:32.192687 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825fd6ae-092f-425f-bc9d-dc217f91f4b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:01:32.193054 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825fd6ae-092f-425f-bc9d-dc217f91f4b8 admin admin] PATH_INFO: `/v3/projects/5e4bd17a5b5746529774561dabdf2bce/users/febb11871c494fada9228617bf1f5650/roles/df8f9f844b1b439181d75a07c1a93418` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:01:32.215165 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825fd6ae-092f-425f-bc9d-dc217f91f4b8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5e4bd17a5b5746529774561dabdf2bce, user_id=febb11871c494fada9228617bf1f5650, role_id=df8f9f844b1b439181d75a07c1a93418)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:01:32.218865 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825fd6ae-092f-425f-bc9d-dc217f91f4b8 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:01:32.243253 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1196/2369] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 30 16:01:32 2022] PUT /identity/v3/projects/5e4bd17a5b5746529774561dabdf2bce/users/febb11871c494fada9228617bf1f5650/roles/df8f9f844b1b439181d75a07c1a93418 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:01:32.255176 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101f8a63-f639-40e4-be87-ff39fcea4e96 None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:01:32.255544 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101f8a63-f639-40e4-be87-ff39fcea4e96 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:01:32.255925 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101f8a63-f639-40e4-be87-ff39fcea4e96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:01:32.309123 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-101f8a63-f639-40e4-be87-ff39fcea4e96 None None] MFA Rules not processed for user `febb11871c494fada9228617bf1f5650`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:01:32.310582 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-101f8a63-f639-40e4-be87-ff39fcea4e96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:01:32.417923 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1174/2370] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 16:01:32 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:01:32.418742 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825fd6ae-092f-425f-bc9d-dc217f91f4b8 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:01:32.527350 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:01:32.538751 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825fd6ae-092f-425f-bc9d-dc217f91f4b8 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:01:32.545952 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7def7368-cd91-432d-aaf3-180689f9a9d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:01:32.546569 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7def7368-cd91-432d-aaf3-180689f9a9d3 admin admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:01:32.546848 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7def7368-cd91-432d-aaf3-180689f9a9d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:01:32.547115 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7def7368-cd91-432d-aaf3-180689f9a9d3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:01:32.547773 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7def7368-cd91-432d-aaf3-180689f9a9d3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:01:32.549780 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7def7368-cd91-432d-aaf3-180689f9a9d3 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:01:32.583496 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1197/2371] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Aug 30 16:01:32 2022] POST /identity/v3/projects => generated 377 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:01:32.585532 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101f8a63-f639-40e4-be87-ff39fcea4e96 None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:01:32.598390 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101f8a63-f639-40e4-be87-ff39fcea4e96 None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:01:32.603884 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:01:32.723807 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a88d864-0611-4080-9479-1febb30bc801 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:01:32.724531 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a88d864-0611-4080-9479-1febb30bc801 admin admin] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:01:32.724832 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a88d864-0611-4080-9479-1febb30bc801 admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:01:32.726187 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a88d864-0611-4080-9479-1febb30bc801 admin admin] PATH_INFO: `/v3/users` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:01:32.726187 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a88d864-0611-4080-9479-1febb30bc801 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:01:32.728255 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a88d864-0611-4080-9479-1febb30bc801 admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:01:32.764570 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1175/2372] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Aug 30 16:01:32 2022] POST /identity/v3/users => generated 295 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:01:32.776375 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7def7368-cd91-432d-aaf3-180689f9a9d3 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:01:32.793326 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7def7368-cd91-432d-aaf3-180689f9a9d3 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:01:32.802727 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e72654-1c6b-443c-aff4-198dce510600 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:01:32.803525 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e72654-1c6b-443c-aff4-198dce510600 admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:01:32.803807 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e72654-1c6b-443c-aff4-198dce510600 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:01:32.804086 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e72654-1c6b-443c-aff4-198dce510600 admin admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:01:32.805529 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e72654-1c6b-443c-aff4-198dce510600 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:01:32.807401 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e72654-1c6b-443c-aff4-198dce510600 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:01:32.836896 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1198/2373] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 30 16:01:32 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 16:01:32.841094 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a88d864-0611-4080-9479-1febb30bc801 admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:01:32.853979 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a88d864-0611-4080-9479-1febb30bc801 admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:01:32.867064 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143c1eeb-fb6b-4500-89f1-4ba503f8ff4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:01:32.867764 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143c1eeb-fb6b-4500-89f1-4ba503f8ff4a admin admin] REQUEST_METHOD: `PUT` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:01:32.868046 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143c1eeb-fb6b-4500-89f1-4ba503f8ff4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:01:32.868323 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143c1eeb-fb6b-4500-89f1-4ba503f8ff4a admin admin] PATH_INFO: `/v3/projects/f5cbcb1f544e4404a729e8db7705ca1d/users/22976b44ca51456ea0bab33875652a8d/roles/f265386950a641ac8413b53eda4ae34d` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:01:32.913094 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143c1eeb-fb6b-4500-89f1-4ba503f8ff4a admin admin] RBAC: Authorizing `identity:create_grant(project_id=f5cbcb1f544e4404a729e8db7705ca1d, user_id=22976b44ca51456ea0bab33875652a8d, role_id=f265386950a641ac8413b53eda4ae34d)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:01:32.915829 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143c1eeb-fb6b-4500-89f1-4ba503f8ff4a admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:01:32.939678 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1176/2374] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 30 16:01:32 2022] PUT /identity/v3/projects/f5cbcb1f544e4404a729e8db7705ca1d/users/22976b44ca51456ea0bab33875652a8d/roles/f265386950a641ac8413b53eda4ae34d => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 16:01:32.943849 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e72654-1c6b-443c-aff4-198dce510600 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:01:32.959509 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e72654-1c6b-443c-aff4-198dce510600 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:01:33.057268 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73cb57d-4108-49c5-9357-d2e28758de1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:01:33.058253 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73cb57d-4108-49c5-9357-d2e28758de1e admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:01:33.058872 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73cb57d-4108-49c5-9357-d2e28758de1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:01:33.059295 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73cb57d-4108-49c5-9357-d2e28758de1e admin admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:01:33.060105 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73cb57d-4108-49c5-9357-d2e28758de1e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:01:33.062448 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73cb57d-4108-49c5-9357-d2e28758de1e admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:01:33.075418 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1199/2375] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 30 16:01:32 2022] GET /identity/v3/roles => generated 1420 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:01:33.080335 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143c1eeb-fb6b-4500-89f1-4ba503f8ff4a admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:01:33.094549 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143c1eeb-fb6b-4500-89f1-4ba503f8ff4a admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:01:33.110121 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc553fd8-d726-4bf0-86b0-e41645236369 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:01:33.110349 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc553fd8-d726-4bf0-86b0-e41645236369 admin admin] REQUEST_METHOD: `PUT` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:01:33.110571 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc553fd8-d726-4bf0-86b0-e41645236369 admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:01:33.111035 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc553fd8-d726-4bf0-86b0-e41645236369 admin admin] PATH_INFO: `/v3/projects/f5cbcb1f544e4404a729e8db7705ca1d/users/22976b44ca51456ea0bab33875652a8d/roles/f265386950a641ac8413b53eda4ae34d` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:01:33.117555 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc553fd8-d726-4bf0-86b0-e41645236369 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f5cbcb1f544e4404a729e8db7705ca1d, user_id=22976b44ca51456ea0bab33875652a8d, role_id=f265386950a641ac8413b53eda4ae34d)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:01:33.120447 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc553fd8-d726-4bf0-86b0-e41645236369 admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:01:33.148648 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1177/2376] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 30 16:01:33 2022] PUT /identity/v3/projects/f5cbcb1f544e4404a729e8db7705ca1d/users/22976b44ca51456ea0bab33875652a8d/roles/f265386950a641ac8413b53eda4ae34d => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 16:01:33.164090 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c6efef-f22f-4025-bf0a-2f42e9333a16 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:01:33.165427 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c6efef-f22f-4025-bf0a-2f42e9333a16 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:01:33.165883 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c6efef-f22f-4025-bf0a-2f42e9333a16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:01:33.260294 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-45c6efef-f22f-4025-bf0a-2f42e9333a16 None None] MFA Rules not processed for user `22976b44ca51456ea0bab33875652a8d`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:01:33.261579 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-45c6efef-f22f-4025-bf0a-2f42e9333a16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:01:33.443116 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1200/2377] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 16:01:33 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 281 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:01:33.455339 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc553fd8-d726-4bf0-86b0-e41645236369 admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:01:33.457699 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-fc553fd8-d726-4bf0-86b0-e41645236369 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:01:33.476115 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add982ef-aa8b-447a-95a1-c2cdd387629e None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:01:33.476722 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add982ef-aa8b-447a-95a1-c2cdd387629e None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:01:33.477523 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add982ef-aa8b-447a-95a1-c2cdd387629e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:01:33.544257 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-add982ef-aa8b-447a-95a1-c2cdd387629e None None] MFA Rules not processed for user `4f00011535aa4836b1cc410dd5d97beb`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:01:33.578491 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-add982ef-aa8b-447a-95a1-c2cdd387629e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:01:33.662891 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc553fd8-d726-4bf0-86b0-e41645236369 admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:01:33.670064 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f616959e-297f-4cd6-a46a-5044c3a81403 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0100a04821804993866ea20f159663f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:01:33.671151 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f616959e-297f-4cd6-a46a-5044c3a81403 service cinder] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:01:33.671151 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f616959e-297f-4cd6-a46a-5044c3a81403 service cinder] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:01:33.671945 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f616959e-297f-4cd6-a46a-5044c3a81403 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:01:33.697876 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:01:33.776486 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1201/2378] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 16:01:33 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 302 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 16:01:33.783544 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add982ef-aa8b-447a-95a1-c2cdd387629e None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:01:33.804644 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-add982ef-aa8b-447a-95a1-c2cdd387629e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:01:33.820113 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f616959e-297f-4cd6-a46a-5044c3a81403 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:01:33.822877 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f616959e-297f-4cd6-a46a-5044c3a81403 service cinder] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:01:33.843692 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1178/2379] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 16:01:33 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:01:33.959133 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add982ef-aa8b-447a-95a1-c2cdd387629e None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:01:33.964405 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0eb14dc-cfff-409c-a6d8-fa7526ae4084 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:01:33.965360 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0eb14dc-cfff-409c-a6d8-fa7526ae4084 admin admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:01:33.965797 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0eb14dc-cfff-409c-a6d8-fa7526ae4084 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:01:33.966122 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0eb14dc-cfff-409c-a6d8-fa7526ae4084 admin admin] PATH_INFO: `/v3/projects` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:01:33.966926 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0eb14dc-cfff-409c-a6d8-fa7526ae4084 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:01:33.969002 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0eb14dc-cfff-409c-a6d8-fa7526ae4084 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:01:34.004613 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1202/2380] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Aug 30 16:01:33 2022] POST /identity/v3/projects => generated 373 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:01:34.008881 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f616959e-297f-4cd6-a46a-5044c3a81403 service cinder] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:01:34.025204 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f616959e-297f-4cd6-a46a-5044c3a81403 service cinder] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:01:34.052772 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:01:34.180969 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965ec6bc-5966-49f3-82db-933bdcb0b749 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:01:34.181701 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965ec6bc-5966-49f3-82db-933bdcb0b749 admin admin] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:01:34.182213 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965ec6bc-5966-49f3-82db-933bdcb0b749 admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:01:34.182556 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965ec6bc-5966-49f3-82db-933bdcb0b749 admin admin] PATH_INFO: `/v3/users` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:01:34.183263 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965ec6bc-5966-49f3-82db-933bdcb0b749 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:01:34.185379 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965ec6bc-5966-49f3-82db-933bdcb0b749 admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:01:34.216037 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1179/2381] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Aug 30 16:01:34 2022] POST /identity/v3/users => generated 293 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:01:34.220085 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0eb14dc-cfff-409c-a6d8-fa7526ae4084 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:01:34.240278 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0eb14dc-cfff-409c-a6d8-fa7526ae4084 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:01:34.249340 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a83378a-b7b6-4c4b-9e00-7e1be7c52279 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:01:34.250204 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a83378a-b7b6-4c4b-9e00-7e1be7c52279 admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:01:34.250428 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a83378a-b7b6-4c4b-9e00-7e1be7c52279 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:01:34.250697 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a83378a-b7b6-4c4b-9e00-7e1be7c52279 admin admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:01:34.251353 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a83378a-b7b6-4c4b-9e00-7e1be7c52279 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:01:34.253495 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a83378a-b7b6-4c4b-9e00-7e1be7c52279 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:01:34.271818 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1203/2382] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 30 16:01:34 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:01:34.276791 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965ec6bc-5966-49f3-82db-933bdcb0b749 admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:01:34.292141 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965ec6bc-5966-49f3-82db-933bdcb0b749 admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:01:34.312773 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eac49ff-9d5a-4bae-bb02-711aa3d9f085 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:01:34.313603 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eac49ff-9d5a-4bae-bb02-711aa3d9f085 admin admin] REQUEST_METHOD: `PUT` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:01:34.313995 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eac49ff-9d5a-4bae-bb02-711aa3d9f085 admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:01:34.314314 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eac49ff-9d5a-4bae-bb02-711aa3d9f085 admin admin] PATH_INFO: `/v3/projects/4ab1c607d41344b18931ea65951813dc/users/43dc309356e14c319e92f55568e93afd/roles/f265386950a641ac8413b53eda4ae34d` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:01:34.335779 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eac49ff-9d5a-4bae-bb02-711aa3d9f085 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4ab1c607d41344b18931ea65951813dc, user_id=43dc309356e14c319e92f55568e93afd, role_id=f265386950a641ac8413b53eda4ae34d)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:01:34.340088 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eac49ff-9d5a-4bae-bb02-711aa3d9f085 admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:01:34.390651 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1180/2383] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 30 16:01:34 2022] PUT /identity/v3/projects/4ab1c607d41344b18931ea65951813dc/users/43dc309356e14c319e92f55568e93afd/roles/f265386950a641ac8413b53eda4ae34d => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:01:34.394977 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a83378a-b7b6-4c4b-9e00-7e1be7c52279 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:01:34.411563 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a83378a-b7b6-4c4b-9e00-7e1be7c52279 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:01:34.517036 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332f4f4b-825c-4016-9e17-6bf0a91af43e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:01:34.518510 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332f4f4b-825c-4016-9e17-6bf0a91af43e admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:01:34.518841 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332f4f4b-825c-4016-9e17-6bf0a91af43e admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:01:34.519041 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332f4f4b-825c-4016-9e17-6bf0a91af43e admin admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:01:34.519766 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332f4f4b-825c-4016-9e17-6bf0a91af43e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:01:34.521776 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332f4f4b-825c-4016-9e17-6bf0a91af43e admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:01:34.536033 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1204/2384] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 30 16:01:34 2022] GET /identity/v3/roles => generated 1420 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:01:34.541096 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eac49ff-9d5a-4bae-bb02-711aa3d9f085 admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:01:34.555368 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eac49ff-9d5a-4bae-bb02-711aa3d9f085 admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:01:34.576592 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01966f79-39c6-4cb5-b79d-d23704fed3d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:01:34.577619 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01966f79-39c6-4cb5-b79d-d23704fed3d5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:01:34.577904 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01966f79-39c6-4cb5-b79d-d23704fed3d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:01:34.578182 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01966f79-39c6-4cb5-b79d-d23704fed3d5 admin admin] PATH_INFO: `/v3/projects/4ab1c607d41344b18931ea65951813dc/users/43dc309356e14c319e92f55568e93afd/roles/f265386950a641ac8413b53eda4ae34d` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:01:34.580444 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01966f79-39c6-4cb5-b79d-d23704fed3d5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4ab1c607d41344b18931ea65951813dc, user_id=43dc309356e14c319e92f55568e93afd, role_id=f265386950a641ac8413b53eda4ae34d)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:01:34.583491 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01966f79-39c6-4cb5-b79d-d23704fed3d5 admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:01:34.594883 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1181/2385] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 30 16:01:34 2022] PUT /identity/v3/projects/4ab1c607d41344b18931ea65951813dc/users/43dc309356e14c319e92f55568e93afd/roles/f265386950a641ac8413b53eda4ae34d => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:01:35.691940 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332f4f4b-825c-4016-9e17-6bf0a91af43e admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:01:35.902652 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:01:35.909852 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332f4f4b-825c-4016-9e17-6bf0a91af43e admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:01:36.053782 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5ba38f-3c19-481a-9bbc-8f27a36c68e7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0100a04821804993866ea20f159663f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:01:36.054755 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5ba38f-3c19-481a-9bbc-8f27a36c68e7 service cinder] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:01:36.055341 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5ba38f-3c19-481a-9bbc-8f27a36c68e7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:01:36.056053 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5ba38f-3c19-481a-9bbc-8f27a36c68e7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:01:36.193099 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5ba38f-3c19-481a-9bbc-8f27a36c68e7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:01:36.201020 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5ba38f-3c19-481a-9bbc-8f27a36c68e7 service cinder] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:01:36.220462 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1205/2386] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 16:01:35 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 529 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:01:36.954868 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a141bb9c-f76f-4700-93e9-2ceca5402593 None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:01:36.955464 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a141bb9c-f76f-4700-93e9-2ceca5402593 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:01:36.955614 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a141bb9c-f76f-4700-93e9-2ceca5402593 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:01:36.994637 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-a141bb9c-f76f-4700-93e9-2ceca5402593 None None] MFA Rules not processed for user `07130d598131443486c2bc1c6673c45b`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:01:36.998270 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-a141bb9c-f76f-4700-93e9-2ceca5402593 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:01:37.154167 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1182/2387] 10.222.0.20 () {58 vars in 1009 bytes} [Tue Aug 30 16:01:36 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:01:37.174685 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5ba38f-3c19-481a-9bbc-8f27a36c68e7 service cinder] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:01:37.176374 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-1b5ba38f-3c19-481a-9bbc-8f27a36c68e7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:01:37.354666 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5ba38f-3c19-481a-9bbc-8f27a36c68e7 service cinder] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:01:37.369947 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a141bb9c-f76f-4700-93e9-2ceca5402593 None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:01:37.382905 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a141bb9c-f76f-4700-93e9-2ceca5402593 None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:01:37.384254 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b307311-fd43-4949-a395-1f51c1d20ff1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8fdafdc39a594bb4b0aeaf2fbc84c2fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:01:37.385133 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b307311-fd43-4949-a395-1f51c1d20ff1 service swift] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:01:37.386384 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b307311-fd43-4949-a395-1f51c1d20ff1 service swift] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:01:37.387219 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b307311-fd43-4949-a395-1f51c1d20ff1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:01:37.393845 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f298663-e933-4555-999d-0b88b1931f8b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0100a04821804993866ea20f159663f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:01:37.397959 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f298663-e933-4555-999d-0b88b1931f8b service cinder] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:01:37.398497 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f298663-e933-4555-999d-0b88b1931f8b service cinder] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:01:37.398973 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f298663-e933-4555-999d-0b88b1931f8b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:01:37.412574 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b307311-fd43-4949-a395-1f51c1d20ff1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:01:37.414656 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b307311-fd43-4949-a395-1f51c1d20ff1 service swift] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:01:37.439064 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1206/2388] 10.222.0.20 () {60 vars in 1384 bytes} [Tue Aug 30 16:01:37 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 264 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 30 16:01:37.562731 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f298663-e933-4555-999d-0b88b1931f8b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:01:37.564512 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f298663-e933-4555-999d-0b88b1931f8b service cinder] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:01:37.633888 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1183/2389] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 16:01:37 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 257 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:01:40.038690 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b435b804-52f1-48bd-8b8d-7ee291183c63 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:01:40.039421 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b435b804-52f1-48bd-8b8d-7ee291183c63 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:01:40.039421 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b435b804-52f1-48bd-8b8d-7ee291183c63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:01:40.092460 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-b435b804-52f1-48bd-8b8d-7ee291183c63 None None] MFA Rules not processed for user `c34836a75f144f74aa3fd6a38ccb9791`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:01:40.094174 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-b435b804-52f1-48bd-8b8d-7ee291183c63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:01:40.190730 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:01:40.229526 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1207/2390] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 16:01:40 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:01:42.496583 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f298663-e933-4555-999d-0b88b1931f8b service cinder] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:01:42.512001 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:01:42.675710 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f298663-e933-4555-999d-0b88b1931f8b service cinder] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:01:42.694028 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd1a274-60c3-4c52-9731-cee24cfc2b1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:01:42.694738 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd1a274-60c3-4c52-9731-cee24cfc2b1b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:01:42.695007 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd1a274-60c3-4c52-9731-cee24cfc2b1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:01:42.695284 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd1a274-60c3-4c52-9731-cee24cfc2b1b admin admin] PATH_INFO: `/v3/users/cb359499a0944dc583a225beb80e0989` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:01:42.696600 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd1a274-60c3-4c52-9731-cee24cfc2b1b admin admin] RBAC: Authorizing `identity:delete_user(user_id=cb359499a0944dc583a225beb80e0989)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:01:42.698608 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd1a274-60c3-4c52-9731-cee24cfc2b1b admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:01:42.898532 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-1bd1a274-60c3-4c52-9731-cee24cfc2b1b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cb359499a0944dc583a225beb80e0989'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:01:42.951371 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-1bd1a274-60c3-4c52-9731-cee24cfc2b1b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cb359499a0944dc583a225beb80e0989'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:01:42.992969 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-1bd1a274-60c3-4c52-9731-cee24cfc2b1b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cb359499a0944dc583a225beb80e0989'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:01:43.032038 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-1bd1a274-60c3-4c52-9731-cee24cfc2b1b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cb359499a0944dc583a225beb80e0989'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:01:43.035042 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-1bd1a274-60c3-4c52-9731-cee24cfc2b1b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cb359499a0944dc583a225beb80e0989'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:01:43.071400 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1184/2391] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Aug 30 16:01:42 2022] DELETE /identity/v3/users/cb359499a0944dc583a225beb80e0989 => generated 0 bytes in 575 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:01:43.085105 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ce6643-70da-4aeb-9afe-d978c10bb1a2 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:01:43.085105 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ce6643-70da-4aeb-9afe-d978c10bb1a2 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:01:43.085105 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ce6643-70da-4aeb-9afe-d978c10bb1a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:01:43.134324 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-e4ce6643-70da-4aeb-9afe-d978c10bb1a2 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:01:43.142879 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-e4ce6643-70da-4aeb-9afe-d978c10bb1a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:01:43.316912 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1208/2392] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 16:01:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:01:43.345795 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd1a274-60c3-4c52-9731-cee24cfc2b1b admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:01:43.347942 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-1bd1a274-60c3-4c52-9731-cee24cfc2b1b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:01:43.481696 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd1a274-60c3-4c52-9731-cee24cfc2b1b admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:01:43.485666 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ff0da2-415a-4fb5-a54a-cf6b8da344f4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f4157105f3488f807b7475008ab722', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:01:43.486387 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ff0da2-415a-4fb5-a54a-cf6b8da344f4 service neutron] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:01:43.487101 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ff0da2-415a-4fb5-a54a-cf6b8da344f4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:01:43.487470 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ff0da2-415a-4fb5-a54a-cf6b8da344f4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:01:43.509798 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ff0da2-415a-4fb5-a54a-cf6b8da344f4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:01:43.511370 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ff0da2-415a-4fb5-a54a-cf6b8da344f4 service neutron] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:01:43.538955 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1185/2393] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 16:01:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:01:44.895864 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ce6643-70da-4aeb-9afe-d978c10bb1a2 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:01:44.898905 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-e4ce6643-70da-4aeb-9afe-d978c10bb1a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:01:44.923706 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ce6643-70da-4aeb-9afe-d978c10bb1a2 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:01:44.931471 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f1f238-79db-492f-8d21-16207c3f759f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:01:44.932342 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f1f238-79db-492f-8d21-16207c3f759f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:01:44.932923 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f1f238-79db-492f-8d21-16207c3f759f admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:01:44.933403 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f1f238-79db-492f-8d21-16207c3f759f admin admin] PATH_INFO: `/v3/projects/1622471368b54226a194dc3ea18db4c2` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:01:44.935644 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f1f238-79db-492f-8d21-16207c3f759f admin admin] RBAC: Authorizing `identity:delete_project(project_id=1622471368b54226a194dc3ea18db4c2)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:01:44.938589 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f1f238-79db-492f-8d21-16207c3f759f admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:01:45.075578 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-98f1f238-79db-492f-8d21-16207c3f759f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1622471368b54226a194dc3ea18db4c2', 'request_id': 'req-98f1f238-79db-492f-8d21-16207c3f759f'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:01:45.091716 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-98f1f238-79db-492f-8d21-16207c3f759f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1622471368b54226a194dc3ea18db4c2', 'request_id': 'req-98f1f238-79db-492f-8d21-16207c3f759f'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:01:45.103550 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-98f1f238-79db-492f-8d21-16207c3f759f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1622471368b54226a194dc3ea18db4c2', 'request_id': 'req-98f1f238-79db-492f-8d21-16207c3f759f'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:01:45.128895 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-98f1f238-79db-492f-8d21-16207c3f759f admin admin] The token cache is being invalidate because project 1622471368b54226a194dc3ea18db4c2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81511) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 16:01:45.129648 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-98f1f238-79db-492f-8d21-16207c3f759f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:01:45.131125 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-98f1f238-79db-492f-8d21-16207c3f759f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:01:45.133808 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1209/2394] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Aug 30 16:01:44 2022] DELETE /identity/v3/projects/1622471368b54226a194dc3ea18db4c2 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 16:01:45.137992 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ff0da2-415a-4fb5-a54a-cf6b8da344f4 service neutron] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:01:45.157831 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ff0da2-415a-4fb5-a54a-cf6b8da344f4 service neutron] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:01:45.213410 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:01:45.262591 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b899d9-14df-4111-af8d-60cd7ad4ddb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:01:45.263591 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b899d9-14df-4111-af8d-60cd7ad4ddb1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:01:45.263948 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b899d9-14df-4111-af8d-60cd7ad4ddb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:01:45.264652 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b899d9-14df-4111-af8d-60cd7ad4ddb1 admin admin] PATH_INFO: `/v3/users/febb11871c494fada9228617bf1f5650` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:01:45.266410 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b899d9-14df-4111-af8d-60cd7ad4ddb1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=febb11871c494fada9228617bf1f5650)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:01:45.269894 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b899d9-14df-4111-af8d-60cd7ad4ddb1 admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:01:45.375624 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-a5b899d9-14df-4111-af8d-60cd7ad4ddb1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'febb11871c494fada9228617bf1f5650'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:01:45.399241 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-a5b899d9-14df-4111-af8d-60cd7ad4ddb1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'febb11871c494fada9228617bf1f5650'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:01:45.421068 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-a5b899d9-14df-4111-af8d-60cd7ad4ddb1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'febb11871c494fada9228617bf1f5650'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:01:45.435710 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-a5b899d9-14df-4111-af8d-60cd7ad4ddb1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'febb11871c494fada9228617bf1f5650'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:01:45.439015 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-a5b899d9-14df-4111-af8d-60cd7ad4ddb1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'febb11871c494fada9228617bf1f5650'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:01:45.467476 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1186/2395] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Aug 30 16:01:45 2022] DELETE /identity/v3/users/febb11871c494fada9228617bf1f5650 => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:01:46.278122 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f1f238-79db-492f-8d21-16207c3f759f admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:01:46.282242 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-98f1f238-79db-492f-8d21-16207c3f759f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:01:46.308592 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f1f238-79db-492f-8d21-16207c3f759f admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:01:46.473948 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efca3223-00e8-4734-a8d1-e70b6148323d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:01:46.475050 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efca3223-00e8-4734-a8d1-e70b6148323d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:01:46.476312 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efca3223-00e8-4734-a8d1-e70b6148323d admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:01:46.476806 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efca3223-00e8-4734-a8d1-e70b6148323d admin admin] PATH_INFO: `/v3/projects/5e4bd17a5b5746529774561dabdf2bce` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:01:46.481087 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efca3223-00e8-4734-a8d1-e70b6148323d admin admin] RBAC: Authorizing `identity:delete_project(project_id=5e4bd17a5b5746529774561dabdf2bce)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:01:46.484739 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efca3223-00e8-4734-a8d1-e70b6148323d admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:01:46.519708 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:01:46.574556 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-efca3223-00e8-4734-a8d1-e70b6148323d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5e4bd17a5b5746529774561dabdf2bce', 'request_id': 'req-efca3223-00e8-4734-a8d1-e70b6148323d'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:01:46.596998 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-efca3223-00e8-4734-a8d1-e70b6148323d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5e4bd17a5b5746529774561dabdf2bce', 'request_id': 'req-efca3223-00e8-4734-a8d1-e70b6148323d'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:01:46.635276 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-efca3223-00e8-4734-a8d1-e70b6148323d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5e4bd17a5b5746529774561dabdf2bce', 'request_id': 'req-efca3223-00e8-4734-a8d1-e70b6148323d'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:01:46.670962 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-efca3223-00e8-4734-a8d1-e70b6148323d admin admin] The token cache is being invalidate because project 5e4bd17a5b5746529774561dabdf2bce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81511) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 16:01:46.674182 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-efca3223-00e8-4734-a8d1-e70b6148323d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:01:46.675414 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-efca3223-00e8-4734-a8d1-e70b6148323d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:01:46.677277 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1210/2396] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Aug 30 16:01:46 2022] DELETE /identity/v3/projects/5e4bd17a5b5746529774561dabdf2bce => generated 0 bytes in 402 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:01:46.681641 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b899d9-14df-4111-af8d-60cd7ad4ddb1 admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:01:46.683327 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-a5b899d9-14df-4111-af8d-60cd7ad4ddb1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:01:46.725774 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b899d9-14df-4111-af8d-60cd7ad4ddb1 admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:01:46.897837 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e288092-ea92-4f1f-9baa-d9446881db70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:01:46.897837 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e288092-ea92-4f1f-9baa-d9446881db70 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:01:46.909210 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e288092-ea92-4f1f-9baa-d9446881db70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:01:46.909210 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e288092-ea92-4f1f-9baa-d9446881db70 admin admin] PATH_INFO: `/v3/users/22976b44ca51456ea0bab33875652a8d` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:01:46.917834 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e288092-ea92-4f1f-9baa-d9446881db70 admin admin] RBAC: Authorizing `identity:delete_user(user_id=22976b44ca51456ea0bab33875652a8d)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:01:46.921295 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e288092-ea92-4f1f-9baa-d9446881db70 admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:01:47.031364 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-4e288092-ea92-4f1f-9baa-d9446881db70 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '22976b44ca51456ea0bab33875652a8d'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:01:47.042565 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:01:47.058321 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-4e288092-ea92-4f1f-9baa-d9446881db70 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '22976b44ca51456ea0bab33875652a8d'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:01:47.080206 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-4e288092-ea92-4f1f-9baa-d9446881db70 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '22976b44ca51456ea0bab33875652a8d'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:01:47.093503 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-4e288092-ea92-4f1f-9baa-d9446881db70 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '22976b44ca51456ea0bab33875652a8d'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:01:47.094407 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-4e288092-ea92-4f1f-9baa-d9446881db70 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '22976b44ca51456ea0bab33875652a8d'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:01:47.103286 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1187/2397] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Aug 30 16:01:46 2022] DELETE /identity/v3/users/22976b44ca51456ea0bab33875652a8d => generated 0 bytes in 422 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:01:47.665904 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efca3223-00e8-4734-a8d1-e70b6148323d admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:01:47.667983 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-efca3223-00e8-4734-a8d1-e70b6148323d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:01:47.701624 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efca3223-00e8-4734-a8d1-e70b6148323d admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:01:47.729138 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a45c039-1d3d-4018-9a84-6926cfcd2941 None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:01:47.730380 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a45c039-1d3d-4018-9a84-6926cfcd2941 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:01:47.731030 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a45c039-1d3d-4018-9a84-6926cfcd2941 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:01:47.768866 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-0a45c039-1d3d-4018-9a84-6926cfcd2941 None None] MFA Rules not processed for user `43dc309356e14c319e92f55568e93afd`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:01:47.770688 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-0a45c039-1d3d-4018-9a84-6926cfcd2941 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:01:47.910958 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d11a99e-21a5-42fe-8f90-b48fd7bb82d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:01:47.911928 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d11a99e-21a5-42fe-8f90-b48fd7bb82d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:01:47.912371 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d11a99e-21a5-42fe-8f90-b48fd7bb82d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:01:47.912791 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d11a99e-21a5-42fe-8f90-b48fd7bb82d0 admin admin] PATH_INFO: `/v3/projects/f5cbcb1f544e4404a729e8db7705ca1d` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:01:47.913430 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1188/2398] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 16:01:47 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:01:47.915118 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d11a99e-21a5-42fe-8f90-b48fd7bb82d0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f5cbcb1f544e4404a729e8db7705ca1d)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:01:47.918776 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d11a99e-21a5-42fe-8f90-b48fd7bb82d0 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:01:47.927386 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a45c039-1d3d-4018-9a84-6926cfcd2941 None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:01:47.930490 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-0a45c039-1d3d-4018-9a84-6926cfcd2941 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:01:48.027719 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-6d11a99e-21a5-42fe-8f90-b48fd7bb82d0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f5cbcb1f544e4404a729e8db7705ca1d', 'request_id': 'req-6d11a99e-21a5-42fe-8f90-b48fd7bb82d0'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:01:48.052177 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-6d11a99e-21a5-42fe-8f90-b48fd7bb82d0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f5cbcb1f544e4404a729e8db7705ca1d', 'request_id': 'req-6d11a99e-21a5-42fe-8f90-b48fd7bb82d0'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:01:48.064287 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-6d11a99e-21a5-42fe-8f90-b48fd7bb82d0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f5cbcb1f544e4404a729e8db7705ca1d', 'request_id': 'req-6d11a99e-21a5-42fe-8f90-b48fd7bb82d0'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:01:48.081827 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-6d11a99e-21a5-42fe-8f90-b48fd7bb82d0 admin admin] The token cache is being invalidate because project f5cbcb1f544e4404a729e8db7705ca1d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81511) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 16:01:48.082682 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-6d11a99e-21a5-42fe-8f90-b48fd7bb82d0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:01:48.083686 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-6d11a99e-21a5-42fe-8f90-b48fd7bb82d0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:01:48.085539 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1211/2399] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Aug 30 16:01:47 2022] DELETE /identity/v3/projects/f5cbcb1f544e4404a729e8db7705ca1d => generated 0 bytes in 420 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:01:48.094092 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f100c1b9-15fc-44df-829a-fa4ac1f6a072 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:01:48.096355 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f100c1b9-15fc-44df-829a-fa4ac1f6a072 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:01:48.096355 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f100c1b9-15fc-44df-829a-fa4ac1f6a072 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:01:48.116327 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a45c039-1d3d-4018-9a84-6926cfcd2941 None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:01:48.125428 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-f100c1b9-15fc-44df-829a-fa4ac1f6a072 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:01:48.127349 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-f100c1b9-15fc-44df-829a-fa4ac1f6a072 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:01:48.260479 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:01:48.312162 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:DELETE=3,keystone:SELECT=97 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:01:48.328299 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1212/2400] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 16:01:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:01:48.330326 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bb8c18-ad0c-4c99-9c56-56ecf35d0ff6 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:01:48.330766 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bb8c18-ad0c-4c99-9c56-56ecf35d0ff6 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:01:48.331103 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bb8c18-ad0c-4c99-9c56-56ecf35d0ff6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:01:48.373552 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308f7d6d-a9d0-485b-8a7e-a852b5921251 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a1e756ae9be4a7bac1e81acf07e8f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:01:48.374185 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308f7d6d-a9d0-485b-8a7e-a852b5921251 service nova] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:01:48.374524 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308f7d6d-a9d0-485b-8a7e-a852b5921251 service nova] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:01:48.374839 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308f7d6d-a9d0-485b-8a7e-a852b5921251 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:01:48.383000 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-308f7d6d-a9d0-485b-8a7e-a852b5921251 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:01:48.386704 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-d3bb8c18-ad0c-4c99-9c56-56ecf35d0ff6 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:01:48.388039 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-d3bb8c18-ad0c-4c99-9c56-56ecf35d0ff6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:01:48.409410 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1213/2401] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 16:01:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 16:01:48.415954 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bb8c18-ad0c-4c99-9c56-56ecf35d0ff6 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:01:48.439899 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bb8c18-ad0c-4c99-9c56-56ecf35d0ff6 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:01:48.453014 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3575504f-7d6b-4571-9f75-549cef3dbdd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:01:48.454189 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3575504f-7d6b-4571-9f75-549cef3dbdd9 admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:01:48.456590 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3575504f-7d6b-4571-9f75-549cef3dbdd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:01:48.456847 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3575504f-7d6b-4571-9f75-549cef3dbdd9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:01:48.457901 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3575504f-7d6b-4571-9f75-549cef3dbdd9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:01:48.457901 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3575504f-7d6b-4571-9f75-549cef3dbdd9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:01:48.461403 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3575504f-7d6b-4571-9f75-549cef3dbdd9 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:01:48.491409 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1214/2402] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Aug 30 16:01:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:01:48.494687 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3575504f-7d6b-4571-9f75-549cef3dbdd9 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:01:48.514624 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3575504f-7d6b-4571-9f75-549cef3dbdd9 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:01:48.522865 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad26dde-088b-4ea8-b73d-f6dbb73b3351 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:01:48.523930 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad26dde-088b-4ea8-b73d-f6dbb73b3351 admin admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:01:48.524733 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad26dde-088b-4ea8-b73d-f6dbb73b3351 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:01:48.525281 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad26dde-088b-4ea8-b73d-f6dbb73b3351 admin admin] PATH_INFO: `/v3/projects` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:01:48.526144 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad26dde-088b-4ea8-b73d-f6dbb73b3351 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:01:48.531804 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad26dde-088b-4ea8-b73d-f6dbb73b3351 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:01:48.570657 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308f7d6d-a9d0-485b-8a7e-a852b5921251 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:01:48.576269 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308f7d6d-a9d0-485b-8a7e-a852b5921251 service nova] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:01:48.580155 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1215/2403] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Aug 30 16:01:48 2022] POST /identity/v3/projects => generated 369 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:01:48.584785 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad26dde-088b-4ea8-b73d-f6dbb73b3351 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:01:48.604381 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1189/2404] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 16:01:47 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 677 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:01:48.700579 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad26dde-088b-4ea8-b73d-f6dbb73b3351 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:01:48.705388 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7dc68e-290c-4071-be6c-9fe0ea816d0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:01:48.706628 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7dc68e-290c-4071-be6c-9fe0ea816d0e admin admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:01:48.707078 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7dc68e-290c-4071-be6c-9fe0ea816d0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:01:48.707354 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7dc68e-290c-4071-be6c-9fe0ea816d0e admin admin] PATH_INFO: `/v3/users` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:01:48.708139 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7dc68e-290c-4071-be6c-9fe0ea816d0e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:01:48.710276 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7dc68e-290c-4071-be6c-9fe0ea816d0e admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:01:48.749842 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1216/2405] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Aug 30 16:01:48 2022] POST /identity/v3/users => generated 285 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:01:48.753988 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308f7d6d-a9d0-485b-8a7e-a852b5921251 service nova] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:01:48.768359 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308f7d6d-a9d0-485b-8a7e-a852b5921251 service nova] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:01:48.786206 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8de128-7da3-442f-9f1b-583699e44294 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:01:48.786886 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8de128-7da3-442f-9f1b-583699e44294 admin admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:01:48.787168 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8de128-7da3-442f-9f1b-583699e44294 admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:01:48.789093 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8de128-7da3-442f-9f1b-583699e44294 admin admin] PATH_INFO: `/v3/roles` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:01:48.792179 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8de128-7da3-442f-9f1b-583699e44294 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:01:48.793859 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8de128-7da3-442f-9f1b-583699e44294 admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:01:48.822844 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7dc68e-290c-4071-be6c-9fe0ea816d0e admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:01:48.825783 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1190/2406] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 30 16:01:48 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 16:01:48.839359 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7dc68e-290c-4071-be6c-9fe0ea816d0e admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:01:48.851069 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f726dcf-b25c-4fca-a5cc-11677575305b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:01:48.851827 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f726dcf-b25c-4fca-a5cc-11677575305b admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:01:48.852155 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f726dcf-b25c-4fca-a5cc-11677575305b admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:01:48.852488 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f726dcf-b25c-4fca-a5cc-11677575305b admin admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:01:48.853309 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f726dcf-b25c-4fca-a5cc-11677575305b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:01:48.855110 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f726dcf-b25c-4fca-a5cc-11677575305b admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:01:48.881859 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8de128-7da3-442f-9f1b-583699e44294 admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:01:48.889514 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1217/2407] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 30 16:01:48 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 16:01:48.894795 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8de128-7da3-442f-9f1b-583699e44294 admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:01:48.908911 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caad9d66-a037-45e7-944a-086b97008b97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:01:48.909734 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caad9d66-a037-45e7-944a-086b97008b97 admin admin] REQUEST_METHOD: `PUT` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:01:48.910096 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caad9d66-a037-45e7-944a-086b97008b97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:01:48.910457 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caad9d66-a037-45e7-944a-086b97008b97 admin admin] PATH_INFO: `/v3/projects/a814626ebef547dca758413f06789ff7/users/0f85096bb19a441ebd00d788e8ca9e54/roles/df8f9f844b1b439181d75a07c1a93418` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:01:48.963765 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caad9d66-a037-45e7-944a-086b97008b97 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a814626ebef547dca758413f06789ff7, user_id=0f85096bb19a441ebd00d788e8ca9e54, role_id=df8f9f844b1b439181d75a07c1a93418)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:01:48.972805 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caad9d66-a037-45e7-944a-086b97008b97 admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:01:49.002954 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1191/2408] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 30 16:01:48 2022] PUT /identity/v3/projects/a814626ebef547dca758413f06789ff7/users/0f85096bb19a441ebd00d788e8ca9e54/roles/df8f9f844b1b439181d75a07c1a93418 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:01:49.016068 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56a826c-6586-405c-a605-86617709bfc4 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:01:49.016574 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56a826c-6586-405c-a605-86617709bfc4 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:01:49.017024 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56a826c-6586-405c-a605-86617709bfc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:01:49.078829 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-e56a826c-6586-405c-a605-86617709bfc4 None None] MFA Rules not processed for user `0f85096bb19a441ebd00d788e8ca9e54`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:01:49.080844 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-e56a826c-6586-405c-a605-86617709bfc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:01:49.225762 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1218/2409] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 16:01:49 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:01:49.246728 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caad9d66-a037-45e7-944a-086b97008b97 admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:01:49.248592 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-caad9d66-a037-45e7-944a-086b97008b97 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:01:49.367829 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caad9d66-a037-45e7-944a-086b97008b97 admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:01:49.385570 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0feb1f-6a22-4c7e-84b1-13bcf5c39b8c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b70e06a7d06d4199ab4fc36342ee1d68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:01:49.387765 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0feb1f-6a22-4c7e-84b1-13bcf5c39b8c service glance] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:01:49.388836 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0feb1f-6a22-4c7e-84b1-13bcf5c39b8c service glance] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:01:49.389413 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0feb1f-6a22-4c7e-84b1-13bcf5c39b8c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:01:49.406654 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0feb1f-6a22-4c7e-84b1-13bcf5c39b8c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:01:49.410719 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0feb1f-6a22-4c7e-84b1-13bcf5c39b8c service glance] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:01:49.432962 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1192/2410] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 16:01:49 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:01:51.228557 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c77049-4bdf-483b-9873-a8eeb4d4bddf None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:01:51.231795 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c77049-4bdf-483b-9873-a8eeb4d4bddf None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:01:51.232258 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c77049-4bdf-483b-9873-a8eeb4d4bddf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:01:51.256611 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:01:51.278814 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-17c77049-4bdf-483b-9873-a8eeb4d4bddf None None] MFA Rules not processed for user `4f00011535aa4836b1cc410dd5d97beb`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:01:51.284063 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-17c77049-4bdf-483b-9873-a8eeb4d4bddf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:01:51.369761 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1219/2411] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 16:01:51 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:01:59.426650 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=74,keystone:INSERT=1 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:02:01.353728 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=23 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:02:01.677823 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4489139f-f50c-42c4-bfa4-748dae23fccf None None] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:02:01.678433 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4489139f-f50c-42c4-bfa4-748dae23fccf None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:02:01.678433 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4489139f-f50c-42c4-bfa4-748dae23fccf None None] PATH_INFO: `/` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:02:01.679982 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1193/2412] 10.222.0.20 () {54 vars in 921 bytes} [Tue Aug 30 16:02:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 16:02:01.685842 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866775b5-0d64-4f49-8d7b-234ddbbfdf5c None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:02:01.686373 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866775b5-0d64-4f49-8d7b-234ddbbfdf5c None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:02:01.686650 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866775b5-0d64-4f49-8d7b-234ddbbfdf5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:02:01.711444 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-866775b5-0d64-4f49-8d7b-234ddbbfdf5c None None] MFA Rules not processed for user `7a1e756ae9be4a7bac1e81acf07e8f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:02:01.712805 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-866775b5-0d64-4f49-8d7b-234ddbbfdf5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:02:01.788075 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1220/2413] 10.222.0.20 () {58 vars in 1018 bytes} [Tue Aug 30 16:02:01 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:02:04.608751 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4489139f-f50c-42c4-bfa4-748dae23fccf None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:02:04.621196 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4489139f-f50c-42c4-bfa4-748dae23fccf None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:02:04.626669 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b13b9cf-02bf-448c-89b0-9de1abd259cd service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b70e06a7d06d4199ab4fc36342ee1d68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:02:04.627346 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b13b9cf-02bf-448c-89b0-9de1abd259cd service glance] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:02:04.627747 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b13b9cf-02bf-448c-89b0-9de1abd259cd service glance] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:02:04.628103 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b13b9cf-02bf-448c-89b0-9de1abd259cd service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:02:04.639388 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b13b9cf-02bf-448c-89b0-9de1abd259cd service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:02:04.640833 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b13b9cf-02bf-448c-89b0-9de1abd259cd service glance] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:02:04.652271 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1194/2414] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 16:02:04 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:02:04.784707 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60269db5-6e8e-4886-8eff-392b850a4d3e None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:02:04.785205 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60269db5-6e8e-4886-8eff-392b850a4d3e None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:02:04.785415 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60269db5-6e8e-4886-8eff-392b850a4d3e None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:02:04.786781 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1221/2415] 10.222.0.20 () {54 vars in 921 bytes} [Tue Aug 30 16:02:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 16:02:04.791600 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8294694d-d811-4a17-a1bd-4e39ec1464c2 None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:02:04.791888 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8294694d-d811-4a17-a1bd-4e39ec1464c2 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:02:04.792216 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8294694d-d811-4a17-a1bd-4e39ec1464c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:02:04.813355 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-8294694d-d811-4a17-a1bd-4e39ec1464c2 None None] MFA Rules not processed for user `65f4157105f3488f807b7475008ab722`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:02:04.814810 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-8294694d-d811-4a17-a1bd-4e39ec1464c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:02:04.868612 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1195/2416] 10.222.0.20 () {58 vars in 1018 bytes} [Tue Aug 30 16:02:04 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:02:04.880800 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60269db5-6e8e-4886-8eff-392b850a4d3e None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:02:04.882685 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-60269db5-6e8e-4886-8eff-392b850a4d3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:02:04.897390 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60269db5-6e8e-4886-8eff-392b850a4d3e None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:02:04.901037 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840f7d7f-f934-4e24-91e9-d336fdba6a44 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f4157105f3488f807b7475008ab722', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:02:04.901723 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840f7d7f-f934-4e24-91e9-d336fdba6a44 service neutron] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:02:04.902001 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840f7d7f-f934-4e24-91e9-d336fdba6a44 service neutron] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:02:04.902271 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840f7d7f-f934-4e24-91e9-d336fdba6a44 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:02:04.910024 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840f7d7f-f934-4e24-91e9-d336fdba6a44 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:02:04.911459 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840f7d7f-f934-4e24-91e9-d336fdba6a44 service neutron] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:02:04.919524 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1222/2417] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 16:02:04 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:02:05.006914 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50994b36-952c-4fdd-9605-1e3e6af063f8 None None] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:02:05.007203 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50994b36-952c-4fdd-9605-1e3e6af063f8 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:02:05.007472 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50994b36-952c-4fdd-9605-1e3e6af063f8 None None] PATH_INFO: `/` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:02:05.008916 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1196/2418] 10.222.0.20 () {54 vars in 921 bytes} [Tue Aug 30 16:02:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 16:02:05.014797 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db6a2c3-743e-431f-bdea-08e0fbd535be None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:02:05.015084 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db6a2c3-743e-431f-bdea-08e0fbd535be None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:02:05.015355 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db6a2c3-743e-431f-bdea-08e0fbd535be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:02:05.036483 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-0db6a2c3-743e-431f-bdea-08e0fbd535be None None] MFA Rules not processed for user `7a1e756ae9be4a7bac1e81acf07e8f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:02:05.037668 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-0db6a2c3-743e-431f-bdea-08e0fbd535be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:02:05.049900 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1223/2419] 10.222.0.20 () {58 vars in 1018 bytes} [Tue Aug 30 16:02:05 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:02:05.061048 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50994b36-952c-4fdd-9605-1e3e6af063f8 None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:02:05.063194 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-50994b36-952c-4fdd-9605-1e3e6af063f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:02:05.078335 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50994b36-952c-4fdd-9605-1e3e6af063f8 None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:02:05.082182 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9856645b-d874-4e9b-90fe-9c69d81fe06d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f4157105f3488f807b7475008ab722', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:02:05.082779 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9856645b-d874-4e9b-90fe-9c69d81fe06d service neutron] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:02:05.083057 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9856645b-d874-4e9b-90fe-9c69d81fe06d service neutron] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:02:05.083325 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9856645b-d874-4e9b-90fe-9c69d81fe06d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:02:05.094393 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9856645b-d874-4e9b-90fe-9c69d81fe06d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:02:05.096059 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9856645b-d874-4e9b-90fe-9c69d81fe06d service neutron] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:02:05.107450 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1197/2420] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 16:02:05 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:02:09.542725 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fbe8a4-6946-4c20-8bad-785744569f3b None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:02:09.543281 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fbe8a4-6946-4c20-8bad-785744569f3b None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:02:09.543281 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fbe8a4-6946-4c20-8bad-785744569f3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:02:09.562565 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-e3fbe8a4-6946-4c20-8bad-785744569f3b None None] MFA Rules not processed for user `07130d598131443486c2bc1c6673c45b`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:02:09.563778 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-e3fbe8a4-6946-4c20-8bad-785744569f3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:02:09.620674 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1224/2421] 10.222.0.20 () {58 vars in 1009 bytes} [Tue Aug 30 16:02:09 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:02:09.637935 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9856645b-d874-4e9b-90fe-9c69d81fe06d service neutron] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:02:09.639776 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-9856645b-d874-4e9b-90fe-9c69d81fe06d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:02:09.718755 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9856645b-d874-4e9b-90fe-9c69d81fe06d service neutron] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:02:09.725724 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d557d8-7d7e-4258-bcf8-d682446c7775 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8fdafdc39a594bb4b0aeaf2fbc84c2fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:02:09.726430 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d557d8-7d7e-4258-bcf8-d682446c7775 service swift] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:02:09.726783 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d557d8-7d7e-4258-bcf8-d682446c7775 service swift] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:02:09.727118 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d557d8-7d7e-4258-bcf8-d682446c7775 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:02:09.737575 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d557d8-7d7e-4258-bcf8-d682446c7775 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:02:09.739239 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d557d8-7d7e-4258-bcf8-d682446c7775 service swift] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:02:09.747937 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1198/2422] 10.222.0.20 () {60 vars in 1384 bytes} [Tue Aug 30 16:02:09 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 30 16:02:19.441094 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a73b65d-6d33-49b7-92a5-ae8fe261fcc8 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:02:19.441818 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a73b65d-6d33-49b7-92a5-ae8fe261fcc8 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:02:19.441818 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a73b65d-6d33-49b7-92a5-ae8fe261fcc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:02:19.470254 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-3a73b65d-6d33-49b7-92a5-ae8fe261fcc8 None None] MFA Rules not processed for user `07130d598131443486c2bc1c6673c45b`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:02:19.471506 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-3a73b65d-6d33-49b7-92a5-ae8fe261fcc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:02:19.485757 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1225/2423] 10.222.0.20 () {58 vars in 1009 bytes} [Tue Aug 30 16:02:19 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:02:19.500530 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d557d8-7d7e-4258-bcf8-d682446c7775 service swift] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:02:19.530440 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d557d8-7d7e-4258-bcf8-d682446c7775 service swift] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:02:19.531649 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d676a1-deef-4739-bfad-db8fd1035a3a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8fdafdc39a594bb4b0aeaf2fbc84c2fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:02:19.532001 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d676a1-deef-4739-bfad-db8fd1035a3a service swift] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:02:19.532857 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d676a1-deef-4739-bfad-db8fd1035a3a service swift] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:02:19.535312 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d676a1-deef-4739-bfad-db8fd1035a3a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:02:19.555753 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d676a1-deef-4739-bfad-db8fd1035a3a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:02:19.557490 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d676a1-deef-4739-bfad-db8fd1035a3a service swift] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:02:19.569733 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1199/2424] 10.222.0.20 () {60 vars in 1384 bytes} [Tue Aug 30 16:02:19 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 70 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 30 16:02:22.300270 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a73b65d-6d33-49b7-92a5-ae8fe261fcc8 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:02:22.302590 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-3a73b65d-6d33-49b7-92a5-ae8fe261fcc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:02:22.552776 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a73b65d-6d33-49b7-92a5-ae8fe261fcc8 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:02:22.560546 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1825125-9ad0-4eb7-9b26-257d55d75481 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc12c6d31eab4a17a160250adc2380c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:02:22.561483 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1825125-9ad0-4eb7-9b26-257d55d75481 service placement] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:02:22.562215 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1825125-9ad0-4eb7-9b26-257d55d75481 service placement] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:02:22.562800 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1825125-9ad0-4eb7-9b26-257d55d75481 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:02:22.567167 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-b1825125-9ad0-4eb7-9b26-257d55d75481 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:02:22.598403 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1825125-9ad0-4eb7-9b26-257d55d75481 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:02:22.600384 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1825125-9ad0-4eb7-9b26-257d55d75481 service placement] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:02:22.619461 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:02:22.645909 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1226/2425] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 16:02:22 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:02:29.564551 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=73 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:02:31.912822 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1aaf451-091c-4670-abfa-4de7a38780ff None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:02:31.913516 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1aaf451-091c-4670-abfa-4de7a38780ff None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:02:31.913516 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1aaf451-091c-4670-abfa-4de7a38780ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:02:31.934181 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-d1aaf451-091c-4670-abfa-4de7a38780ff None None] MFA Rules not processed for user `07130d598131443486c2bc1c6673c45b`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:02:31.935383 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-d1aaf451-091c-4670-abfa-4de7a38780ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:02:31.943650 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1200/2426] 10.222.0.20 () {58 vars in 967 bytes} [Tue Aug 30 16:02:31 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:02:31.956170 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1825125-9ad0-4eb7-9b26-257d55d75481 service placement] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:02:31.967825 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1825125-9ad0-4eb7-9b26-257d55d75481 service placement] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:02:31.973141 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeea57e4-d630-4324-b640-e238da719e9a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8fdafdc39a594bb4b0aeaf2fbc84c2fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:02:31.973888 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeea57e4-d630-4324-b640-e238da719e9a service swift] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:02:31.974166 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeea57e4-d630-4324-b640-e238da719e9a service swift] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:02:31.974441 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeea57e4-d630-4324-b640-e238da719e9a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:02:31.982774 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeea57e4-d630-4324-b640-e238da719e9a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:02:31.984367 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeea57e4-d630-4324-b640-e238da719e9a service swift] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:02:31.993188 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1227/2427] 10.222.0.20 () {60 vars in 1384 bytes} [Tue Aug 30 16:02:31 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 30 16:02:33.981015 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1aaf451-091c-4670-abfa-4de7a38780ff None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:02:33.983162 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-d1aaf451-091c-4670-abfa-4de7a38780ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:02:34.048975 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1aaf451-091c-4670-abfa-4de7a38780ff None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:02:34.053483 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085b8954-d0a4-428b-bf6c-a21967abfa16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:02:34.054088 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085b8954-d0a4-428b-bf6c-a21967abfa16 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:02:34.054368 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085b8954-d0a4-428b-bf6c-a21967abfa16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:02:34.054640 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085b8954-d0a4-428b-bf6c-a21967abfa16 admin admin] PATH_INFO: `/v3/users/e9fdf5c5f263474bb41ab41bdac6f78f` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:02:34.055798 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085b8954-d0a4-428b-bf6c-a21967abfa16 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e9fdf5c5f263474bb41ab41bdac6f78f)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:02:34.057787 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085b8954-d0a4-428b-bf6c-a21967abfa16 admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:02:34.111152 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-085b8954-d0a4-428b-bf6c-a21967abfa16 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e9fdf5c5f263474bb41ab41bdac6f78f'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:02:34.120155 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-085b8954-d0a4-428b-bf6c-a21967abfa16 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e9fdf5c5f263474bb41ab41bdac6f78f'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:02:34.132300 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-085b8954-d0a4-428b-bf6c-a21967abfa16 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e9fdf5c5f263474bb41ab41bdac6f78f'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:02:34.139245 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-085b8954-d0a4-428b-bf6c-a21967abfa16 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e9fdf5c5f263474bb41ab41bdac6f78f'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:02:34.139917 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-085b8954-d0a4-428b-bf6c-a21967abfa16 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e9fdf5c5f263474bb41ab41bdac6f78f'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:02:34.145847 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1201/2428] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Aug 30 16:02:33 2022] DELETE /identity/v3/users/e9fdf5c5f263474bb41ab41bdac6f78f => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:02:34.329068 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeea57e4-d630-4324-b640-e238da719e9a service swift] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:02:34.340927 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeea57e4-d630-4324-b640-e238da719e9a service swift] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:02:34.355855 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085b8954-d0a4-428b-bf6c-a21967abfa16 admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:02:34.357555 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-085b8954-d0a4-428b-bf6c-a21967abfa16 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:02:34.408384 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b28fc7-b3ee-4b60-90b8-bd7312103b30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:02:34.409022 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b28fc7-b3ee-4b60-90b8-bd7312103b30 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:02:34.409364 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b28fc7-b3ee-4b60-90b8-bd7312103b30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:02:34.409681 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b28fc7-b3ee-4b60-90b8-bd7312103b30 admin admin] PATH_INFO: `/v3/projects/80493f357d57418cb58f8a33e82b8822` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:02:34.410837 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b28fc7-b3ee-4b60-90b8-bd7312103b30 admin admin] RBAC: Authorizing `identity:delete_project(project_id=80493f357d57418cb58f8a33e82b8822)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:02:34.412753 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b28fc7-b3ee-4b60-90b8-bd7312103b30 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:02:34.427919 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085b8954-d0a4-428b-bf6c-a21967abfa16 admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:02:34.432495 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde5a2f3-978f-4883-80b2-e495e67e0b55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:02:34.433114 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde5a2f3-978f-4883-80b2-e495e67e0b55 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:02:34.433423 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde5a2f3-978f-4883-80b2-e495e67e0b55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:02:34.433690 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde5a2f3-978f-4883-80b2-e495e67e0b55 admin admin] PATH_INFO: `/v3/users/0f85096bb19a441ebd00d788e8ca9e54` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:02:34.434791 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde5a2f3-978f-4883-80b2-e495e67e0b55 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0f85096bb19a441ebd00d788e8ca9e54)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:02:34.436714 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde5a2f3-978f-4883-80b2-e495e67e0b55 admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:02:34.451428 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:02:34.461529 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-d0b28fc7-b3ee-4b60-90b8-bd7312103b30 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '80493f357d57418cb58f8a33e82b8822', 'request_id': 'req-d0b28fc7-b3ee-4b60-90b8-bd7312103b30'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:02:34.471455 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-d0b28fc7-b3ee-4b60-90b8-bd7312103b30 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '80493f357d57418cb58f8a33e82b8822', 'request_id': 'req-d0b28fc7-b3ee-4b60-90b8-bd7312103b30'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:02:34.478015 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-d0b28fc7-b3ee-4b60-90b8-bd7312103b30 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '80493f357d57418cb58f8a33e82b8822', 'request_id': 'req-d0b28fc7-b3ee-4b60-90b8-bd7312103b30'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:02:34.489030 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-d0b28fc7-b3ee-4b60-90b8-bd7312103b30 admin admin] The token cache is being invalidate because project 80493f357d57418cb58f8a33e82b8822 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81511) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 16:02:34.489176 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-dde5a2f3-978f-4883-80b2-e495e67e0b55 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0f85096bb19a441ebd00d788e8ca9e54'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:02:34.489328 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-d0b28fc7-b3ee-4b60-90b8-bd7312103b30 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:02:34.489957 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-d0b28fc7-b3ee-4b60-90b8-bd7312103b30 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:02:34.491496 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1228/2429] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Aug 30 16:02:34 2022] DELETE /identity/v3/projects/80493f357d57418cb58f8a33e82b8822 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:02:34.498345 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-dde5a2f3-978f-4883-80b2-e495e67e0b55 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0f85096bb19a441ebd00d788e8ca9e54'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:02:34.512693 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-dde5a2f3-978f-4883-80b2-e495e67e0b55 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0f85096bb19a441ebd00d788e8ca9e54'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:02:34.520616 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-dde5a2f3-978f-4883-80b2-e495e67e0b55 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0f85096bb19a441ebd00d788e8ca9e54'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:02:34.521448 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-dde5a2f3-978f-4883-80b2-e495e67e0b55 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0f85096bb19a441ebd00d788e8ca9e54'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:02:34.527957 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1202/2430] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Aug 30 16:02:34 2022] DELETE /identity/v3/users/0f85096bb19a441ebd00d788e8ca9e54 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:02:34.533002 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee538769-30b6-4692-b531-d5f122b3a031 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:02:34.533244 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee538769-30b6-4692-b531-d5f122b3a031 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:02:34.533533 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee538769-30b6-4692-b531-d5f122b3a031 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:02:34.554995 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-ee538769-30b6-4692-b531-d5f122b3a031 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:02:34.556235 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-ee538769-30b6-4692-b531-d5f122b3a031 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:02:34.625738 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1229/2431] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 16:02:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:02:34.643212 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde5a2f3-978f-4883-80b2-e495e67e0b55 admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:02:34.644972 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-dde5a2f3-978f-4883-80b2-e495e67e0b55 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:02:34.707065 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde5a2f3-978f-4883-80b2-e495e67e0b55 admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:02:34.710862 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef824b5-1ac1-4c6b-835a-e3759a667e1a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f4157105f3488f807b7475008ab722', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:02:34.711484 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef824b5-1ac1-4c6b-835a-e3759a667e1a service neutron] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:02:34.711757 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef824b5-1ac1-4c6b-835a-e3759a667e1a service neutron] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:02:34.712024 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef824b5-1ac1-4c6b-835a-e3759a667e1a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:02:34.721871 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef824b5-1ac1-4c6b-835a-e3759a667e1a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:02:34.723359 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef824b5-1ac1-4c6b-835a-e3759a667e1a service neutron] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:02:34.733473 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1203/2432] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 16:02:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:02:35.094415 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee538769-30b6-4692-b531-d5f122b3a031 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:02:35.095900 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-ee538769-30b6-4692-b531-d5f122b3a031 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:02:35.112698 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee538769-30b6-4692-b531-d5f122b3a031 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:02:35.118402 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4fe3f2-0c33-414a-a932-49fb8e36bfab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:02:35.119409 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4fe3f2-0c33-414a-a932-49fb8e36bfab admin admin] REQUEST_METHOD: `DELETE` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:02:35.119692 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4fe3f2-0c33-414a-a932-49fb8e36bfab admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:02:35.119988 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4fe3f2-0c33-414a-a932-49fb8e36bfab admin admin] PATH_INFO: `/v3/projects/a814626ebef547dca758413f06789ff7` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:02:35.121248 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4fe3f2-0c33-414a-a932-49fb8e36bfab admin admin] RBAC: Authorizing `identity:delete_project(project_id=a814626ebef547dca758413f06789ff7)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:02:35.123219 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4fe3f2-0c33-414a-a932-49fb8e36bfab admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:02:35.136352 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:02:35.175115 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-4e4fe3f2-0c33-414a-a932-49fb8e36bfab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a814626ebef547dca758413f06789ff7', 'request_id': 'req-4e4fe3f2-0c33-414a-a932-49fb8e36bfab'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:02:35.185101 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-4e4fe3f2-0c33-414a-a932-49fb8e36bfab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a814626ebef547dca758413f06789ff7', 'request_id': 'req-4e4fe3f2-0c33-414a-a932-49fb8e36bfab'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:02:35.191921 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-4e4fe3f2-0c33-414a-a932-49fb8e36bfab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a814626ebef547dca758413f06789ff7', 'request_id': 'req-4e4fe3f2-0c33-414a-a932-49fb8e36bfab'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:02:35.202470 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-4e4fe3f2-0c33-414a-a932-49fb8e36bfab admin admin] The token cache is being invalidate because project a814626ebef547dca758413f06789ff7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81511) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 16:02:35.202784 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-4e4fe3f2-0c33-414a-a932-49fb8e36bfab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:02:35.203430 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-4e4fe3f2-0c33-414a-a932-49fb8e36bfab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:02:35.204940 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1230/2433] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Aug 30 16:02:35 2022] DELETE /identity/v3/projects/a814626ebef547dca758413f06789ff7 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:02:35.210439 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e729a2-75ba-4984-9835-3a59fe804a64 None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:02:35.210992 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e729a2-75ba-4984-9835-3a59fe804a64 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:02:35.210992 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e729a2-75ba-4984-9835-3a59fe804a64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:02:35.231392 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-73e729a2-75ba-4984-9835-3a59fe804a64 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:02:35.232603 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-73e729a2-75ba-4984-9835-3a59fe804a64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:02:35.294354 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1204/2434] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 16:02:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:02:35.302582 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4665db94-3308-431c-ad52-528be2bf7231 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:02:35.302845 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4665db94-3308-431c-ad52-528be2bf7231 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:02:35.303205 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4665db94-3308-431c-ad52-528be2bf7231 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:02:35.323693 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-4665db94-3308-431c-ad52-528be2bf7231 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:02:35.324875 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-4665db94-3308-431c-ad52-528be2bf7231 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:02:35.333253 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1231/2435] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 16:02:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:02:35.337913 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e729a2-75ba-4984-9835-3a59fe804a64 None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:02:35.348170 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e729a2-75ba-4984-9835-3a59fe804a64 None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:02:35.353634 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e03bf53-a2d6-48f4-8216-2f855851fc67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:02:35.354232 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e03bf53-a2d6-48f4-8216-2f855851fc67 admin admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:02:35.354506 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e03bf53-a2d6-48f4-8216-2f855851fc67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:02:35.354770 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e03bf53-a2d6-48f4-8216-2f855851fc67 admin admin] PATH_INFO: `/v3/domains` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:02:35.355437 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e03bf53-a2d6-48f4-8216-2f855851fc67 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81512) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:02:35.355716 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e03bf53-a2d6-48f4-8216-2f855851fc67 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:02:35.357359 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e03bf53-a2d6-48f4-8216-2f855851fc67 admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:02:35.369948 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1205/2436] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Aug 30 16:02:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:02:35.374428 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4665db94-3308-431c-ad52-528be2bf7231 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:02:35.384321 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4665db94-3308-431c-ad52-528be2bf7231 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:02:35.389683 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bcd5c1-b818-4803-b01c-008de4d9d5ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:02:35.390219 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bcd5c1-b818-4803-b01c-008de4d9d5ff admin admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:02:35.390500 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bcd5c1-b818-4803-b01c-008de4d9d5ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:02:35.390771 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bcd5c1-b818-4803-b01c-008de4d9d5ff admin admin] PATH_INFO: `/v3/projects` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:02:35.391404 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39bcd5c1-b818-4803-b01c-008de4d9d5ff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:02:35.393359 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39bcd5c1-b818-4803-b01c-008de4d9d5ff admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:02:35.434061 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1232/2437] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Aug 30 16:02:35 2022] POST /identity/v3/projects => generated 363 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:02:35.440209 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e03bf53-a2d6-48f4-8216-2f855851fc67 admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:02:35.451289 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e03bf53-a2d6-48f4-8216-2f855851fc67 admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:02:35.471379 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:02:35.513894 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7f7449-28e0-4e5d-9b81-9d0b3fe8846f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:02:35.514498 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7f7449-28e0-4e5d-9b81-9d0b3fe8846f admin admin] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:02:35.514777 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7f7449-28e0-4e5d-9b81-9d0b3fe8846f admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:02:35.515046 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7f7449-28e0-4e5d-9b81-9d0b3fe8846f admin admin] PATH_INFO: `/v3/users` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:02:35.515671 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7f7449-28e0-4e5d-9b81-9d0b3fe8846f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:02:35.517634 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7f7449-28e0-4e5d-9b81-9d0b3fe8846f admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:02:35.543640 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1206/2438] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Aug 30 16:02:35 2022] POST /identity/v3/users => generated 282 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:02:35.549467 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bcd5c1-b818-4803-b01c-008de4d9d5ff admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:02:35.560799 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bcd5c1-b818-4803-b01c-008de4d9d5ff admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:02:35.566789 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a708f0fc-b92d-4bc6-b214-f3441024bf1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:02:35.567399 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a708f0fc-b92d-4bc6-b214-f3441024bf1f admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:02:35.567683 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a708f0fc-b92d-4bc6-b214-f3441024bf1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:02:35.567949 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a708f0fc-b92d-4bc6-b214-f3441024bf1f admin admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:02:35.568599 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a708f0fc-b92d-4bc6-b214-f3441024bf1f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:02:35.570624 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a708f0fc-b92d-4bc6-b214-f3441024bf1f admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:02:35.581264 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1233/2439] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 30 16:02:35 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:02:35.585817 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7f7449-28e0-4e5d-9b81-9d0b3fe8846f admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:02:35.600789 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7f7449-28e0-4e5d-9b81-9d0b3fe8846f admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:02:35.609682 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18968b92-4dc3-4507-a0c4-670dd6fbe59a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:02:35.610522 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18968b92-4dc3-4507-a0c4-670dd6fbe59a admin admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:02:35.611197 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18968b92-4dc3-4507-a0c4-670dd6fbe59a admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:02:35.611509 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18968b92-4dc3-4507-a0c4-670dd6fbe59a admin admin] PATH_INFO: `/v3/roles` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:02:35.612514 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18968b92-4dc3-4507-a0c4-670dd6fbe59a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:02:35.614197 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18968b92-4dc3-4507-a0c4-670dd6fbe59a admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:02:35.625472 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1207/2440] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 30 16:02:35 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 16:02:35.629676 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a708f0fc-b92d-4bc6-b214-f3441024bf1f admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:02:35.640847 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a708f0fc-b92d-4bc6-b214-f3441024bf1f admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:02:35.646658 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49b7247-4774-46f4-af79-d7c1468dc6cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:02:35.647526 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49b7247-4774-46f4-af79-d7c1468dc6cf admin admin] REQUEST_METHOD: `PUT` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:02:35.647643 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49b7247-4774-46f4-af79-d7c1468dc6cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:02:35.647846 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49b7247-4774-46f4-af79-d7c1468dc6cf admin admin] PATH_INFO: `/v3/projects/2a7189700f0d4476a555e25130c5f255/users/6736e81744e946c7853d700baa5cfd42/roles/df8f9f844b1b439181d75a07c1a93418` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:02:35.663124 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49b7247-4774-46f4-af79-d7c1468dc6cf admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a7189700f0d4476a555e25130c5f255, user_id=6736e81744e946c7853d700baa5cfd42, role_id=df8f9f844b1b439181d75a07c1a93418)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:02:35.665891 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49b7247-4774-46f4-af79-d7c1468dc6cf admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:02:35.680445 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1234/2441] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 30 16:02:35 2022] PUT /identity/v3/projects/2a7189700f0d4476a555e25130c5f255/users/6736e81744e946c7853d700baa5cfd42/roles/df8f9f844b1b439181d75a07c1a93418 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:02:35.693542 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e7bdf6-44c0-42b2-aaf1-4fecea9597a7 None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:02:35.693828 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e7bdf6-44c0-42b2-aaf1-4fecea9597a7 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:02:35.694096 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e7bdf6-44c0-42b2-aaf1-4fecea9597a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:02:35.723787 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-c4e7bdf6-44c0-42b2-aaf1-4fecea9597a7 None None] MFA Rules not processed for user `6736e81744e946c7853d700baa5cfd42`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:02:35.725014 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-c4e7bdf6-44c0-42b2-aaf1-4fecea9597a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:02:35.790763 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1208/2442] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 16:02:35 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:02:35.803557 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49b7247-4774-46f4-af79-d7c1468dc6cf admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:02:35.805401 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-e49b7247-4774-46f4-af79-d7c1468dc6cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:02:35.863260 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49b7247-4774-46f4-af79-d7c1468dc6cf admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:02:35.866972 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70551860-12bc-48a3-86d2-d4414d92b1cd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0100a04821804993866ea20f159663f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:02:35.867862 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70551860-12bc-48a3-86d2-d4414d92b1cd service cinder] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:02:35.868140 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70551860-12bc-48a3-86d2-d4414d92b1cd service cinder] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:02:35.868402 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70551860-12bc-48a3-86d2-d4414d92b1cd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:02:35.876868 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70551860-12bc-48a3-86d2-d4414d92b1cd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:02:35.878472 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70551860-12bc-48a3-86d2-d4414d92b1cd service cinder] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:02:35.887985 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1235/2443] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 16:02:35 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:02:45.787538 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=56,keystone:INSERT=3 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:02:45.883853 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=72,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:02:48.616651 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb12d9f-41a3-40a3-9fb8-b2b427062895 None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:02:48.617481 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb12d9f-41a3-40a3-9fb8-b2b427062895 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:02:48.617974 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb12d9f-41a3-40a3-9fb8-b2b427062895 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:02:48.638847 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-7fb12d9f-41a3-40a3-9fb8-b2b427062895 None None] MFA Rules not processed for user `07130d598131443486c2bc1c6673c45b`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:02:48.640828 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-7fb12d9f-41a3-40a3-9fb8-b2b427062895 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:02:48.704510 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1209/2444] 10.222.0.20 () {58 vars in 967 bytes} [Tue Aug 30 16:02:48 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:02:48.719511 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70551860-12bc-48a3-86d2-d4414d92b1cd service cinder] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:02:48.721582 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-70551860-12bc-48a3-86d2-d4414d92b1cd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:02:48.794672 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70551860-12bc-48a3-86d2-d4414d92b1cd service cinder] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:02:48.799927 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f3a851-e0f3-48e9-ac45-582fa413bb02 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8fdafdc39a594bb4b0aeaf2fbc84c2fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:02:48.800540 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f3a851-e0f3-48e9-ac45-582fa413bb02 service swift] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:02:48.800841 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f3a851-e0f3-48e9-ac45-582fa413bb02 service swift] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:02:48.801109 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f3a851-e0f3-48e9-ac45-582fa413bb02 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:02:48.810134 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f3a851-e0f3-48e9-ac45-582fa413bb02 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:02:48.811765 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f3a851-e0f3-48e9-ac45-582fa413bb02 service swift] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:02:48.820755 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1236/2445] 10.222.0.20 () {60 vars in 1384 bytes} [Tue Aug 30 16:02:48 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 30 16:02:55.504010 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb12d9f-41a3-40a3-9fb8-b2b427062895 None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:02:55.506320 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-7fb12d9f-41a3-40a3-9fb8-b2b427062895 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:02:55.584907 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb12d9f-41a3-40a3-9fb8-b2b427062895 None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:02:55.588826 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3664db-a083-4b0a-a5ea-9abae5ca3725 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f4157105f3488f807b7475008ab722', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:02:55.589476 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3664db-a083-4b0a-a5ea-9abae5ca3725 service neutron] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:02:55.589758 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3664db-a083-4b0a-a5ea-9abae5ca3725 service neutron] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:02:55.590036 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3664db-a083-4b0a-a5ea-9abae5ca3725 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:02:55.592027 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-fa3664db-a083-4b0a-a5ea-9abae5ca3725 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:02:55.661656 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3664db-a083-4b0a-a5ea-9abae5ca3725 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:02:55.663328 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3664db-a083-4b0a-a5ea-9abae5ca3725 service neutron] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:02:55.674340 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1210/2446] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 16:02:55 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:02:58.817137 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=32 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:03:03.186452 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdfe9da-15bd-4cde-a899-d7dff38cb7a3 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:03:03.187126 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdfe9da-15bd-4cde-a899-d7dff38cb7a3 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:03:03.187126 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdfe9da-15bd-4cde-a899-d7dff38cb7a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:03:03.206383 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-bcdfe9da-15bd-4cde-a899-d7dff38cb7a3 None None] MFA Rules not processed for user `b133ae684f20468f982ae0c25020d837`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:03:03.207719 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-bcdfe9da-15bd-4cde-a899-d7dff38cb7a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:03:03.270053 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1237/2447] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 16:03:03 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:03:05.669848 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=75 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:03:12.905973 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3664db-a083-4b0a-a5ea-9abae5ca3725 service neutron] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:03:12.970566 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3664db-a083-4b0a-a5ea-9abae5ca3725 service neutron] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:03:12.975610 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29aaf4b7-63e1-47a6-9c0f-98347241113a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:03:12.976237 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29aaf4b7-63e1-47a6-9c0f-98347241113a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:03:12.976517 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29aaf4b7-63e1-47a6-9c0f-98347241113a admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:03:12.976788 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29aaf4b7-63e1-47a6-9c0f-98347241113a admin admin] PATH_INFO: `/v3/users/6736e81744e946c7853d700baa5cfd42` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:03:12.977880 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29aaf4b7-63e1-47a6-9c0f-98347241113a admin admin] RBAC: Authorizing `identity:delete_user(user_id=6736e81744e946c7853d700baa5cfd42)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:03:12.979839 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29aaf4b7-63e1-47a6-9c0f-98347241113a admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:03:13.037152 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-29aaf4b7-63e1-47a6-9c0f-98347241113a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6736e81744e946c7853d700baa5cfd42'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:03:13.047325 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-29aaf4b7-63e1-47a6-9c0f-98347241113a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6736e81744e946c7853d700baa5cfd42'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:03:13.064821 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-29aaf4b7-63e1-47a6-9c0f-98347241113a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6736e81744e946c7853d700baa5cfd42'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:03:13.072880 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-29aaf4b7-63e1-47a6-9c0f-98347241113a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6736e81744e946c7853d700baa5cfd42'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:03:13.073626 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-29aaf4b7-63e1-47a6-9c0f-98347241113a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6736e81744e946c7853d700baa5cfd42'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:03:13.080733 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1211/2448] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Aug 30 16:03:12 2022] DELETE /identity/v3/users/6736e81744e946c7853d700baa5cfd42 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:03:13.087219 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337a0aba-80d2-4d96-8f74-e0959ba36f83 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:03:13.087974 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337a0aba-80d2-4d96-8f74-e0959ba36f83 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:03:13.088162 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337a0aba-80d2-4d96-8f74-e0959ba36f83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:03:13.110502 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-337a0aba-80d2-4d96-8f74-e0959ba36f83 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:03:13.111723 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-337a0aba-80d2-4d96-8f74-e0959ba36f83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:03:13.178052 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1238/2449] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 16:03:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:03:13.187806 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29aaf4b7-63e1-47a6-9c0f-98347241113a admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:03:13.189485 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-29aaf4b7-63e1-47a6-9c0f-98347241113a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:03:13.246942 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29aaf4b7-63e1-47a6-9c0f-98347241113a admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:03:13.250824 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec654f26-7d47-429f-815a-de56263ec878 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f4157105f3488f807b7475008ab722', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:03:13.251525 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec654f26-7d47-429f-815a-de56263ec878 service neutron] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:03:13.251805 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec654f26-7d47-429f-815a-de56263ec878 service neutron] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:03:13.252073 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec654f26-7d47-429f-815a-de56263ec878 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:03:13.266172 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec654f26-7d47-429f-815a-de56263ec878 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:03:13.267728 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec654f26-7d47-429f-815a-de56263ec878 service neutron] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:03:13.278862 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1212/2450] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 16:03:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:03:13.592819 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337a0aba-80d2-4d96-8f74-e0959ba36f83 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:03:13.594420 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-337a0aba-80d2-4d96-8f74-e0959ba36f83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:03:13.608774 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337a0aba-80d2-4d96-8f74-e0959ba36f83 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:03:13.614009 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d27825-b79a-4a58-bc4b-c44a4c944b0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:03:13.615165 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d27825-b79a-4a58-bc4b-c44a4c944b0e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:03:13.615615 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d27825-b79a-4a58-bc4b-c44a4c944b0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:03:13.615890 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d27825-b79a-4a58-bc4b-c44a4c944b0e admin admin] PATH_INFO: `/v3/projects/2a7189700f0d4476a555e25130c5f255` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:03:13.617038 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d27825-b79a-4a58-bc4b-c44a4c944b0e admin admin] RBAC: Authorizing `identity:delete_project(project_id=2a7189700f0d4476a555e25130c5f255)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:03:13.619006 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d27825-b79a-4a58-bc4b-c44a4c944b0e admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:03:13.678361 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-91d27825-b79a-4a58-bc4b-c44a4c944b0e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2a7189700f0d4476a555e25130c5f255', 'request_id': 'req-91d27825-b79a-4a58-bc4b-c44a4c944b0e'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:03:13.691702 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-91d27825-b79a-4a58-bc4b-c44a4c944b0e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2a7189700f0d4476a555e25130c5f255', 'request_id': 'req-91d27825-b79a-4a58-bc4b-c44a4c944b0e'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:03:13.701551 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-91d27825-b79a-4a58-bc4b-c44a4c944b0e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2a7189700f0d4476a555e25130c5f255', 'request_id': 'req-91d27825-b79a-4a58-bc4b-c44a4c944b0e'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:03:13.713402 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-91d27825-b79a-4a58-bc4b-c44a4c944b0e admin admin] The token cache is being invalidate because project 2a7189700f0d4476a555e25130c5f255 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81511) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 16:03:13.713629 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-91d27825-b79a-4a58-bc4b-c44a4c944b0e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:03:13.714338 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-91d27825-b79a-4a58-bc4b-c44a4c944b0e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:03:13.715860 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1239/2451] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Aug 30 16:03:13 2022] DELETE /identity/v3/projects/2a7189700f0d4476a555e25130c5f255 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:03:23.081526 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8111e8-9633-4032-b846-484998f93793 None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:03:23.082078 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8111e8-9633-4032-b846-484998f93793 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:03:23.082078 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8111e8-9633-4032-b846-484998f93793 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:03:23.112860 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-8c8111e8-9633-4032-b846-484998f93793 None None] MFA Rules not processed for user `c34836a75f144f74aa3fd6a38ccb9791`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:03:23.114127 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-8c8111e8-9633-4032-b846-484998f93793 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:03:23.135261 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:03:23.177841 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1213/2452] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 16:03:23 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:03:23.710322 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=79,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:03:33.170638 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=15 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:03:41.246360 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306dd018-00c3-444a-8286-4aad2f4439f0 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:03:41.246927 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306dd018-00c3-444a-8286-4aad2f4439f0 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:03:41.246927 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306dd018-00c3-444a-8286-4aad2f4439f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:03:41.272481 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-306dd018-00c3-444a-8286-4aad2f4439f0 None None] MFA Rules not processed for user `252dd6215082420cade91b100c73f100`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:03:41.274038 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-306dd018-00c3-444a-8286-4aad2f4439f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:03:41.332721 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1240/2453] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 16:03:41 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:03:51.325576 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=25 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:08.998679 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8111e8-9633-4032-b846-484998f93793 None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:09.000179 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-8c8111e8-9633-4032-b846-484998f93793 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:04:09.060147 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8111e8-9633-4032-b846-484998f93793 None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:09.063910 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcca310b-ce23-4461-97cb-de3fcb39a281 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f4157105f3488f807b7475008ab722', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:09.064519 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcca310b-ce23-4461-97cb-de3fcb39a281 service neutron] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:09.064796 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcca310b-ce23-4461-97cb-de3fcb39a281 service neutron] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:09.065070 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcca310b-ce23-4461-97cb-de3fcb39a281 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:09.066921 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-bcca310b-ce23-4461-97cb-de3fcb39a281 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:04:09.144515 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcca310b-ce23-4461-97cb-de3fcb39a281 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:09.146145 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcca310b-ce23-4461-97cb-de3fcb39a281 service neutron] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:09.158673 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1214/2454] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 16:04:08 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 16:04:10.362963 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306dd018-00c3-444a-8286-4aad2f4439f0 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:10.374749 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306dd018-00c3-444a-8286-4aad2f4439f0 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:10.378675 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2aee2c1-4164-4091-80d0-842e15d5d490 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f4157105f3488f807b7475008ab722', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:10.379487 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2aee2c1-4164-4091-80d0-842e15d5d490 service neutron] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:10.379795 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2aee2c1-4164-4091-80d0-842e15d5d490 service neutron] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:10.380067 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2aee2c1-4164-4091-80d0-842e15d5d490 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:10.385395 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-a2aee2c1-4164-4091-80d0-842e15d5d490 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:04:10.400057 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2aee2c1-4164-4091-80d0-842e15d5d490 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:10.401529 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2aee2c1-4164-4091-80d0-842e15d5d490 service neutron] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:10.413176 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1241/2455] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 16:04:10 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:04:15.806026 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bbcbbf-4dc9-470e-9059-b9b9e25f3dc5 None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:15.806571 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bbcbbf-4dc9-470e-9059-b9b9e25f3dc5 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:15.806571 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bbcbbf-4dc9-470e-9059-b9b9e25f3dc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:15.826260 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-d3bbcbbf-4dc9-470e-9059-b9b9e25f3dc5 None None] MFA Rules not processed for user `b133ae684f20468f982ae0c25020d837`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:04:15.827461 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-d3bbcbbf-4dc9-470e-9059-b9b9e25f3dc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:04:15.886674 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1215/2456] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 16:04:15 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:04:18.981412 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2aee2c1-4164-4091-80d0-842e15d5d490 service neutron] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:18.990890 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2aee2c1-4164-4091-80d0-842e15d5d490 service neutron] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:18.995205 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddd7b12-e410-4e55-a918-7ff3b902174d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f4157105f3488f807b7475008ab722', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:18.995893 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddd7b12-e410-4e55-a918-7ff3b902174d service neutron] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:18.996240 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddd7b12-e410-4e55-a918-7ff3b902174d service neutron] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:18.996579 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddd7b12-e410-4e55-a918-7ff3b902174d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:18.998786 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-1ddd7b12-e410-4e55-a918-7ff3b902174d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:04:19.011727 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ddd7b12-e410-4e55-a918-7ff3b902174d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:19.013389 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ddd7b12-e410-4e55-a918-7ff3b902174d service neutron] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:19.023789 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1242/2457] 10.222.0.20 () {60 vars in 1396 bytes} [Tue Aug 30 16:04:18 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3262 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:04:24.076635 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bbcbbf-4dc9-470e-9059-b9b9e25f3dc5 None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:24.078481 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-d3bbcbbf-4dc9-470e-9059-b9b9e25f3dc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:04:24.096725 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bbcbbf-4dc9-470e-9059-b9b9e25f3dc5 None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:24.103551 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10e316e-66ca-46b5-be1b-daa73c0d1bee service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a1e756ae9be4a7bac1e81acf07e8f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:24.104153 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10e316e-66ca-46b5-be1b-daa73c0d1bee service nova] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:24.104436 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10e316e-66ca-46b5-be1b-daa73c0d1bee service nova] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:24.104705 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10e316e-66ca-46b5-be1b-daa73c0d1bee service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:24.106640 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-e10e316e-66ca-46b5-be1b-daa73c0d1bee service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:04:24.120411 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10e316e-66ca-46b5-be1b-daa73c0d1bee service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:24.122048 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10e316e-66ca-46b5-be1b-daa73c0d1bee service nova] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:24.132674 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1216/2458] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 16:04:24 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:04:29.019557 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=12 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:29.121522 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddd7b12-e410-4e55-a918-7ff3b902174d service neutron] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:29.131610 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddd7b12-e410-4e55-a918-7ff3b902174d service neutron] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:29.142631 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f7a83e-8a9f-4a03-9474-81a0e37471a3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f4157105f3488f807b7475008ab722', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:29.143260 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f7a83e-8a9f-4a03-9474-81a0e37471a3 service neutron] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:29.143564 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f7a83e-8a9f-4a03-9474-81a0e37471a3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:29.143833 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f7a83e-8a9f-4a03-9474-81a0e37471a3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:29.145731 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-21f7a83e-8a9f-4a03-9474-81a0e37471a3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:04:29.216050 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f7a83e-8a9f-4a03-9474-81a0e37471a3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:29.217806 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f7a83e-8a9f-4a03-9474-81a0e37471a3 service neutron] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:29.228139 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1243/2459] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 16:04:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:04:31.220373 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10e316e-66ca-46b5-be1b-daa73c0d1bee service nova] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:31.222187 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-e10e316e-66ca-46b5-be1b-daa73c0d1bee service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:04:31.237619 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10e316e-66ca-46b5-be1b-daa73c0d1bee service nova] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:31.254147 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f1e689-94b1-4158-8c4f-54d835a4a294 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a1e756ae9be4a7bac1e81acf07e8f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:31.254147 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f1e689-94b1-4158-8c4f-54d835a4a294 service nova] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:31.254147 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f1e689-94b1-4158-8c4f-54d835a4a294 service nova] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:31.254147 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f1e689-94b1-4158-8c4f-54d835a4a294 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:31.256038 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-05f1e689-94b1-4158-8c4f-54d835a4a294 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:04:31.269257 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f1e689-94b1-4158-8c4f-54d835a4a294 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:31.271565 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f1e689-94b1-4158-8c4f-54d835a4a294 service nova] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:31.282770 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1217/2460] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 16:04:31 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:04:32.864985 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f7a83e-8a9f-4a03-9474-81a0e37471a3 service neutron] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:32.866619 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-21f7a83e-8a9f-4a03-9474-81a0e37471a3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:04:32.881235 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f7a83e-8a9f-4a03-9474-81a0e37471a3 service neutron] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:32.886443 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372c00b0-c99b-449d-9c5d-d3fd2a183f6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:32.887205 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372c00b0-c99b-449d-9c5d-d3fd2a183f6b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:32.887661 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372c00b0-c99b-449d-9c5d-d3fd2a183f6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:32.888114 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372c00b0-c99b-449d-9c5d-d3fd2a183f6b admin admin] PATH_INFO: `/v3/users/c34836a75f144f74aa3fd6a38ccb9791` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:32.889376 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372c00b0-c99b-449d-9c5d-d3fd2a183f6b admin admin] RBAC: Authorizing `identity:delete_user(user_id=c34836a75f144f74aa3fd6a38ccb9791)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:32.891310 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372c00b0-c99b-449d-9c5d-d3fd2a183f6b admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:32.941346 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-372c00b0-c99b-449d-9c5d-d3fd2a183f6b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c34836a75f144f74aa3fd6a38ccb9791'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:04:32.953523 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-372c00b0-c99b-449d-9c5d-d3fd2a183f6b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c34836a75f144f74aa3fd6a38ccb9791'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:04:32.962359 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-372c00b0-c99b-449d-9c5d-d3fd2a183f6b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c34836a75f144f74aa3fd6a38ccb9791'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:04:32.963074 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-372c00b0-c99b-449d-9c5d-d3fd2a183f6b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c34836a75f144f74aa3fd6a38ccb9791'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:04:32.970135 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-372c00b0-c99b-449d-9c5d-d3fd2a183f6b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c34836a75f144f74aa3fd6a38ccb9791'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:04:32.976109 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1244/2461] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Aug 30 16:04:32 2022] DELETE /identity/v3/users/c34836a75f144f74aa3fd6a38ccb9791 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:04:33.160446 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f1e689-94b1-4158-8c4f-54d835a4a294 service nova] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:33.175047 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f1e689-94b1-4158-8c4f-54d835a4a294 service nova] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:33.180056 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=94 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:33.242285 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573bb5b8-09ca-45f9-b5ce-9b73bf7ee4f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:33.243018 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573bb5b8-09ca-45f9-b5ce-9b73bf7ee4f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:33.243356 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573bb5b8-09ca-45f9-b5ce-9b73bf7ee4f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:33.243691 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573bb5b8-09ca-45f9-b5ce-9b73bf7ee4f5 admin admin] PATH_INFO: `/v3/projects/10a598aa5be54fc29948da53d8ac7a32` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:33.244887 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573bb5b8-09ca-45f9-b5ce-9b73bf7ee4f5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=10a598aa5be54fc29948da53d8ac7a32)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:33.246941 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573bb5b8-09ca-45f9-b5ce-9b73bf7ee4f5 admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:33.304891 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-573bb5b8-09ca-45f9-b5ce-9b73bf7ee4f5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '10a598aa5be54fc29948da53d8ac7a32', 'request_id': 'req-573bb5b8-09ca-45f9-b5ce-9b73bf7ee4f5'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:04:33.316437 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-573bb5b8-09ca-45f9-b5ce-9b73bf7ee4f5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '10a598aa5be54fc29948da53d8ac7a32', 'request_id': 'req-573bb5b8-09ca-45f9-b5ce-9b73bf7ee4f5'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:04:33.335433 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-573bb5b8-09ca-45f9-b5ce-9b73bf7ee4f5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '10a598aa5be54fc29948da53d8ac7a32', 'request_id': 'req-573bb5b8-09ca-45f9-b5ce-9b73bf7ee4f5'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:04:33.349387 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-573bb5b8-09ca-45f9-b5ce-9b73bf7ee4f5 admin admin] The token cache is being invalidate because project 10a598aa5be54fc29948da53d8ac7a32 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81512) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 16:04:33.349387 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-573bb5b8-09ca-45f9-b5ce-9b73bf7ee4f5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:04:33.350157 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-573bb5b8-09ca-45f9-b5ce-9b73bf7ee4f5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:04:33.353876 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1218/2462] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Aug 30 16:04:33 2022] DELETE /identity/v3/projects/10a598aa5be54fc29948da53d8ac7a32 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:04:33.363320 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd68996-bc11-4771-b9aa-e021be500667 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:33.363586 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd68996-bc11-4771-b9aa-e021be500667 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:33.363972 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd68996-bc11-4771-b9aa-e021be500667 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:33.386903 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-ebd68996-bc11-4771-b9aa-e021be500667 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:04:33.388473 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-ebd68996-bc11-4771-b9aa-e021be500667 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:04:33.475432 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1245/2463] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 16:04:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:04:33.483731 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acaac8c3-f320-4892-8000-93de9df5bfb1 None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:33.484579 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acaac8c3-f320-4892-8000-93de9df5bfb1 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:33.486337 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acaac8c3-f320-4892-8000-93de9df5bfb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:33.510815 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-acaac8c3-f320-4892-8000-93de9df5bfb1 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:04:33.512013 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-acaac8c3-f320-4892-8000-93de9df5bfb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:04:33.520134 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1219/2464] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 16:04:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:04:33.524770 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd68996-bc11-4771-b9aa-e021be500667 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:33.535418 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd68996-bc11-4771-b9aa-e021be500667 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:33.540442 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa8acb1-f132-46bc-b5fd-d1368b042148 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:33.541468 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa8acb1-f132-46bc-b5fd-d1368b042148 admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:33.541468 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa8acb1-f132-46bc-b5fd-d1368b042148 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:33.541709 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa8acb1-f132-46bc-b5fd-d1368b042148 admin admin] PATH_INFO: `/v3/domains` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:33.542241 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa8acb1-f132-46bc-b5fd-d1368b042148 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:04:33.542542 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa8acb1-f132-46bc-b5fd-d1368b042148 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:33.544147 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa8acb1-f132-46bc-b5fd-d1368b042148 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:33.545966 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:33.560048 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1246/2465] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Aug 30 16:04:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:04:33.563016 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acaac8c3-f320-4892-8000-93de9df5bfb1 None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:33.572713 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acaac8c3-f320-4892-8000-93de9df5bfb1 None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:33.577497 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c795db-c307-43e3-aae1-51c3d35c6a18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:33.578227 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c795db-c307-43e3-aae1-51c3d35c6a18 admin admin] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:33.578593 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c795db-c307-43e3-aae1-51c3d35c6a18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:33.578958 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c795db-c307-43e3-aae1-51c3d35c6a18 admin admin] PATH_INFO: `/v3/projects` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:33.579707 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c795db-c307-43e3-aae1-51c3d35c6a18 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:33.581726 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c795db-c307-43e3-aae1-51c3d35c6a18 admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:33.619994 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1220/2466] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Aug 30 16:04:33 2022] POST /identity/v3/projects => generated 365 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:04:33.623859 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa8acb1-f132-46bc-b5fd-d1368b042148 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:33.634565 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa8acb1-f132-46bc-b5fd-d1368b042148 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:33.705410 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2102e9-c227-426d-8656-1619c70c0752 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:33.706643 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2102e9-c227-426d-8656-1619c70c0752 admin admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:33.706643 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2102e9-c227-426d-8656-1619c70c0752 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:33.706854 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2102e9-c227-426d-8656-1619c70c0752 admin admin] PATH_INFO: `/v3/users` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:33.707351 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2102e9-c227-426d-8656-1619c70c0752 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:33.709347 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2102e9-c227-426d-8656-1619c70c0752 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:33.736010 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1247/2467] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Aug 30 16:04:33 2022] POST /identity/v3/users => generated 283 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:04:33.740139 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c795db-c307-43e3-aae1-51c3d35c6a18 admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:33.751183 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c795db-c307-43e3-aae1-51c3d35c6a18 admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:33.757177 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34103e2c-6e84-485d-bbec-6ff38ee3c0f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:33.758021 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34103e2c-6e84-485d-bbec-6ff38ee3c0f4 admin admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:33.758360 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34103e2c-6e84-485d-bbec-6ff38ee3c0f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:33.758692 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34103e2c-6e84-485d-bbec-6ff38ee3c0f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:33.759408 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34103e2c-6e84-485d-bbec-6ff38ee3c0f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:33.761088 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34103e2c-6e84-485d-bbec-6ff38ee3c0f4 admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:33.776480 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1221/2468] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 30 16:04:33 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:04:33.777621 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2102e9-c227-426d-8656-1619c70c0752 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:33.791053 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2102e9-c227-426d-8656-1619c70c0752 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:33.798007 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce4579e-2c96-4f17-b389-2af8011bc16d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:33.798920 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce4579e-2c96-4f17-b389-2af8011bc16d admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:33.799374 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce4579e-2c96-4f17-b389-2af8011bc16d admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:33.799719 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce4579e-2c96-4f17-b389-2af8011bc16d admin admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:33.800440 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce4579e-2c96-4f17-b389-2af8011bc16d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:33.802230 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce4579e-2c96-4f17-b389-2af8011bc16d admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:33.817551 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1248/2469] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 30 16:04:33 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 16:04:33.822323 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34103e2c-6e84-485d-bbec-6ff38ee3c0f4 admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:33.836203 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34103e2c-6e84-485d-bbec-6ff38ee3c0f4 admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:33.842781 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5658919-8000-4757-a43a-49c6a01bad51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:33.843499 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5658919-8000-4757-a43a-49c6a01bad51 admin admin] REQUEST_METHOD: `PUT` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:33.843785 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5658919-8000-4757-a43a-49c6a01bad51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:33.844093 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5658919-8000-4757-a43a-49c6a01bad51 admin admin] PATH_INFO: `/v3/projects/ffb52ae3323545129e2a7e0dc84e1c93/users/999c9a4b48d14c428470c64182980df4/roles/df8f9f844b1b439181d75a07c1a93418` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:33.869194 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5658919-8000-4757-a43a-49c6a01bad51 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ffb52ae3323545129e2a7e0dc84e1c93, user_id=999c9a4b48d14c428470c64182980df4, role_id=df8f9f844b1b439181d75a07c1a93418)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:33.875886 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5658919-8000-4757-a43a-49c6a01bad51 admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:33.892759 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1222/2470] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 30 16:04:33 2022] PUT /identity/v3/projects/ffb52ae3323545129e2a7e0dc84e1c93/users/999c9a4b48d14c428470c64182980df4/roles/df8f9f844b1b439181d75a07c1a93418 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:04:33.902988 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d08ddb7-f905-49a8-a5bd-bc3d298c52d3 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:33.903564 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d08ddb7-f905-49a8-a5bd-bc3d298c52d3 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:33.903564 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d08ddb7-f905-49a8-a5bd-bc3d298c52d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:33.934235 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-7d08ddb7-f905-49a8-a5bd-bc3d298c52d3 None None] MFA Rules not processed for user `999c9a4b48d14c428470c64182980df4`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:04:33.935517 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-7d08ddb7-f905-49a8-a5bd-bc3d298c52d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:04:34.008223 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1249/2471] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 16:04:33 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:04:34.012898 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5658919-8000-4757-a43a-49c6a01bad51 admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:34.061077 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:34.096259 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5658919-8000-4757-a43a-49c6a01bad51 admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:34.101416 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8d7885-d72b-49a5-9971-f68a886fbe05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:34.102315 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8d7885-d72b-49a5-9971-f68a886fbe05 admin admin] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:34.102596 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8d7885-d72b-49a5-9971-f68a886fbe05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:34.102865 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8d7885-d72b-49a5-9971-f68a886fbe05 admin admin] PATH_INFO: `/v3/projects` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:34.103496 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8d7885-d72b-49a5-9971-f68a886fbe05 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:34.105607 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8d7885-d72b-49a5-9971-f68a886fbe05 admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:34.142082 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1223/2472] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Aug 30 16:04:34 2022] POST /identity/v3/projects => generated 363 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:04:34.145843 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d08ddb7-f905-49a8-a5bd-bc3d298c52d3 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:34.157163 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d08ddb7-f905-49a8-a5bd-bc3d298c52d3 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:34.222593 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c030a6-984a-4f31-9cee-64725430ae3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:34.223296 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c030a6-984a-4f31-9cee-64725430ae3b admin admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:34.223578 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c030a6-984a-4f31-9cee-64725430ae3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:34.223849 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c030a6-984a-4f31-9cee-64725430ae3b admin admin] PATH_INFO: `/v3/users` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:34.224510 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c030a6-984a-4f31-9cee-64725430ae3b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:34.226514 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c030a6-984a-4f31-9cee-64725430ae3b admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:34.238585 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:34.262387 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8d7885-d72b-49a5-9971-f68a886fbe05 admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:34.273721 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1250/2473] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Aug 30 16:04:34 2022] POST /identity/v3/users => generated 288 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:04:34.274508 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8d7885-d72b-49a5-9971-f68a886fbe05 admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:34.284915 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6921a02-4f0f-4089-ad63-e1e219ec505a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:34.285662 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6921a02-4f0f-4089-ad63-e1e219ec505a admin admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:34.285943 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6921a02-4f0f-4089-ad63-e1e219ec505a admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:34.286220 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6921a02-4f0f-4089-ad63-e1e219ec505a admin admin] PATH_INFO: `/v3/roles` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:34.286872 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6921a02-4f0f-4089-ad63-e1e219ec505a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:34.288491 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6921a02-4f0f-4089-ad63-e1e219ec505a admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:34.302469 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1224/2474] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 30 16:04:34 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:04:34.306554 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c030a6-984a-4f31-9cee-64725430ae3b admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:34.319266 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c030a6-984a-4f31-9cee-64725430ae3b admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:34.324749 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbb8234-a0fe-4019-b7bc-1a2ed717ee66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:34.325562 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbb8234-a0fe-4019-b7bc-1a2ed717ee66 admin admin] REQUEST_METHOD: `PUT` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:34.325910 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbb8234-a0fe-4019-b7bc-1a2ed717ee66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:34.326291 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbb8234-a0fe-4019-b7bc-1a2ed717ee66 admin admin] PATH_INFO: `/v3/projects/50d8da8775684e5c8b41e65e6b1ce59a/users/6c5b2ca617d94379bdbeb3a53e19cdf0/roles/f265386950a641ac8413b53eda4ae34d` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:34.345037 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbb8234-a0fe-4019-b7bc-1a2ed717ee66 admin admin] RBAC: Authorizing `identity:create_grant(project_id=50d8da8775684e5c8b41e65e6b1ce59a, user_id=6c5b2ca617d94379bdbeb3a53e19cdf0, role_id=f265386950a641ac8413b53eda4ae34d)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:34.347830 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbb8234-a0fe-4019-b7bc-1a2ed717ee66 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:34.362898 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1251/2475] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 30 16:04:34 2022] PUT /identity/v3/projects/50d8da8775684e5c8b41e65e6b1ce59a/users/6c5b2ca617d94379bdbeb3a53e19cdf0/roles/f265386950a641ac8413b53eda4ae34d => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:04:34.366985 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6921a02-4f0f-4089-ad63-e1e219ec505a admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:34.377178 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6921a02-4f0f-4089-ad63-e1e219ec505a admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:34.435286 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b38a4b-b35b-427c-8ddf-bfc2e711d6d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:34.436102 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b38a4b-b35b-427c-8ddf-bfc2e711d6d8 admin admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:34.436371 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b38a4b-b35b-427c-8ddf-bfc2e711d6d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:34.436632 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b38a4b-b35b-427c-8ddf-bfc2e711d6d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:34.437301 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b38a4b-b35b-427c-8ddf-bfc2e711d6d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:34.438929 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b38a4b-b35b-427c-8ddf-bfc2e711d6d8 admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:34.452849 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1225/2476] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 30 16:04:34 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:04:34.454149 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbb8234-a0fe-4019-b7bc-1a2ed717ee66 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:34.465053 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbb8234-a0fe-4019-b7bc-1a2ed717ee66 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:34.469947 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbea40ef-ffd1-4c6f-a032-9a3301b445bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:34.470744 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbea40ef-ffd1-4c6f-a032-9a3301b445bb admin admin] REQUEST_METHOD: `PUT` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:34.471114 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbea40ef-ffd1-4c6f-a032-9a3301b445bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:34.471438 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbea40ef-ffd1-4c6f-a032-9a3301b445bb admin admin] PATH_INFO: `/v3/projects/50d8da8775684e5c8b41e65e6b1ce59a/users/6c5b2ca617d94379bdbeb3a53e19cdf0/roles/f265386950a641ac8413b53eda4ae34d` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:34.473494 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbea40ef-ffd1-4c6f-a032-9a3301b445bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=50d8da8775684e5c8b41e65e6b1ce59a, user_id=6c5b2ca617d94379bdbeb3a53e19cdf0, role_id=f265386950a641ac8413b53eda4ae34d)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:34.476154 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbea40ef-ffd1-4c6f-a032-9a3301b445bb admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:34.487964 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1252/2477] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 30 16:04:34 2022] PUT /identity/v3/projects/50d8da8775684e5c8b41e65e6b1ce59a/users/6c5b2ca617d94379bdbeb3a53e19cdf0/roles/f265386950a641ac8413b53eda4ae34d => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 16:04:34.500160 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4879ed1d-42fb-4fe2-a719-70ae33b5e075 None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:34.500488 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4879ed1d-42fb-4fe2-a719-70ae33b5e075 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:34.500746 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4879ed1d-42fb-4fe2-a719-70ae33b5e075 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:34.535117 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-4879ed1d-42fb-4fe2-a719-70ae33b5e075 None None] MFA Rules not processed for user `6c5b2ca617d94379bdbeb3a53e19cdf0`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:04:34.536475 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-4879ed1d-42fb-4fe2-a719-70ae33b5e075 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:04:34.616872 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1226/2478] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 16:04:34 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:04:34.628808 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbea40ef-ffd1-4c6f-a032-9a3301b445bb admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:34.630640 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-cbea40ef-ffd1-4c6f-a032-9a3301b445bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:04:34.694355 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbea40ef-ffd1-4c6f-a032-9a3301b445bb admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:34.698469 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f9f4eb-3dbe-4530-8a85-10e56ffbd180 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0100a04821804993866ea20f159663f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:34.699071 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f9f4eb-3dbe-4530-8a85-10e56ffbd180 service cinder] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:34.699347 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f9f4eb-3dbe-4530-8a85-10e56ffbd180 service cinder] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:34.699619 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f9f4eb-3dbe-4530-8a85-10e56ffbd180 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:34.709241 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f9f4eb-3dbe-4530-8a85-10e56ffbd180 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:34.710698 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f9f4eb-3dbe-4530-8a85-10e56ffbd180 service cinder] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:34.720310 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1253/2479] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 16:04:34 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:04:34.803493 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4879ed1d-42fb-4fe2-a719-70ae33b5e075 None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:34.813783 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4879ed1d-42fb-4fe2-a719-70ae33b5e075 None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:34.818224 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bffad8-4894-454c-9866-b0c6d77d1ec9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0100a04821804993866ea20f159663f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:34.818827 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bffad8-4894-454c-9866-b0c6d77d1ec9 service cinder] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:34.819102 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bffad8-4894-454c-9866-b0c6d77d1ec9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:34.819367 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bffad8-4894-454c-9866-b0c6d77d1ec9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:34.846400 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:34.879628 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2bffad8-4894-454c-9866-b0c6d77d1ec9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:34.881075 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2bffad8-4894-454c-9866-b0c6d77d1ec9 service cinder] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:34.890019 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1227/2480] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 16:04:34 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:04:37.707973 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f9f4eb-3dbe-4530-8a85-10e56ffbd180 service cinder] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:37.771018 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f9f4eb-3dbe-4530-8a85-10e56ffbd180 service cinder] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:37.775778 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cb0c6c-a51a-468a-8ee9-d055128c1b12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:37.776409 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cb0c6c-a51a-468a-8ee9-d055128c1b12 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:37.776719 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cb0c6c-a51a-468a-8ee9-d055128c1b12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:37.776931 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cb0c6c-a51a-468a-8ee9-d055128c1b12 admin admin] PATH_INFO: `/v3/users/999c9a4b48d14c428470c64182980df4` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:37.778101 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77cb0c6c-a51a-468a-8ee9-d055128c1b12 admin admin] RBAC: Authorizing `identity:delete_user(user_id=999c9a4b48d14c428470c64182980df4)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:37.780011 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77cb0c6c-a51a-468a-8ee9-d055128c1b12 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:37.829546 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-77cb0c6c-a51a-468a-8ee9-d055128c1b12 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '999c9a4b48d14c428470c64182980df4'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:04:37.837205 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:INSERT=4,keystone:SELECT=88,keystone:DELETE=8 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:37.841882 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-77cb0c6c-a51a-468a-8ee9-d055128c1b12 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '999c9a4b48d14c428470c64182980df4'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:04:37.851357 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-77cb0c6c-a51a-468a-8ee9-d055128c1b12 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '999c9a4b48d14c428470c64182980df4'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:04:37.851967 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-77cb0c6c-a51a-468a-8ee9-d055128c1b12 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '999c9a4b48d14c428470c64182980df4'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:04:37.860242 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-77cb0c6c-a51a-468a-8ee9-d055128c1b12 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '999c9a4b48d14c428470c64182980df4'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:04:37.866213 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1254/2481] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Aug 30 16:04:37 2022] DELETE /identity/v3/users/999c9a4b48d14c428470c64182980df4 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:04:37.870832 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46795d7-8907-4acb-8cd3-12cd5007b343 None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:37.871315 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46795d7-8907-4acb-8cd3-12cd5007b343 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:37.871411 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46795d7-8907-4acb-8cd3-12cd5007b343 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:37.890209 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-c46795d7-8907-4acb-8cd3-12cd5007b343 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:04:37.891451 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-c46795d7-8907-4acb-8cd3-12cd5007b343 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:04:37.950453 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1228/2482] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 16:04:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:04:37.962017 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cb0c6c-a51a-468a-8ee9-d055128c1b12 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:37.963742 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-77cb0c6c-a51a-468a-8ee9-d055128c1b12 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:04:38.022836 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cb0c6c-a51a-468a-8ee9-d055128c1b12 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:38.026944 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe58e73-0e1c-4802-973d-596b858c5477 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f4157105f3488f807b7475008ab722', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:38.027556 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe58e73-0e1c-4802-973d-596b858c5477 service neutron] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:38.027831 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe58e73-0e1c-4802-973d-596b858c5477 service neutron] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:38.028099 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe58e73-0e1c-4802-973d-596b858c5477 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:38.037438 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe58e73-0e1c-4802-973d-596b858c5477 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:38.038881 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe58e73-0e1c-4802-973d-596b858c5477 service neutron] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:38.048713 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1255/2483] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 16:04:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:04:38.334513 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46795d7-8907-4acb-8cd3-12cd5007b343 None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:38.335977 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-c46795d7-8907-4acb-8cd3-12cd5007b343 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:04:38.350992 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46795d7-8907-4acb-8cd3-12cd5007b343 None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:38.355989 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de11f00c-4df8-4be6-8527-45bea25b9861 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:38.356634 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de11f00c-4df8-4be6-8527-45bea25b9861 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:38.356912 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de11f00c-4df8-4be6-8527-45bea25b9861 admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:38.357184 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de11f00c-4df8-4be6-8527-45bea25b9861 admin admin] PATH_INFO: `/v3/projects/ffb52ae3323545129e2a7e0dc84e1c93` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:38.358784 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de11f00c-4df8-4be6-8527-45bea25b9861 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ffb52ae3323545129e2a7e0dc84e1c93)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:38.360734 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de11f00c-4df8-4be6-8527-45bea25b9861 admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:38.411883 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-de11f00c-4df8-4be6-8527-45bea25b9861 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ffb52ae3323545129e2a7e0dc84e1c93', 'request_id': 'req-de11f00c-4df8-4be6-8527-45bea25b9861'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:04:38.421730 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-de11f00c-4df8-4be6-8527-45bea25b9861 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ffb52ae3323545129e2a7e0dc84e1c93', 'request_id': 'req-de11f00c-4df8-4be6-8527-45bea25b9861'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:04:38.428192 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-de11f00c-4df8-4be6-8527-45bea25b9861 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ffb52ae3323545129e2a7e0dc84e1c93', 'request_id': 'req-de11f00c-4df8-4be6-8527-45bea25b9861'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:04:38.443621 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-de11f00c-4df8-4be6-8527-45bea25b9861 admin admin] The token cache is being invalidate because project ffb52ae3323545129e2a7e0dc84e1c93 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81512) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 16:04:38.444091 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-de11f00c-4df8-4be6-8527-45bea25b9861 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:04:38.445111 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-de11f00c-4df8-4be6-8527-45bea25b9861 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:04:38.447495 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1229/2484] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Aug 30 16:04:38 2022] DELETE /identity/v3/projects/ffb52ae3323545129e2a7e0dc84e1c93 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:04:38.453876 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe58e73-0e1c-4802-973d-596b858c5477 service neutron] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:38.466144 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe58e73-0e1c-4802-973d-596b858c5477 service neutron] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:38.532845 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a96540-1aad-40f9-b595-eaafb826704f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:38.533494 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a96540-1aad-40f9-b595-eaafb826704f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:38.533767 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a96540-1aad-40f9-b595-eaafb826704f admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:38.534032 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a96540-1aad-40f9-b595-eaafb826704f admin admin] PATH_INFO: `/v3/users/6c5b2ca617d94379bdbeb3a53e19cdf0` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:38.535277 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a96540-1aad-40f9-b595-eaafb826704f admin admin] RBAC: Authorizing `identity:delete_user(user_id=6c5b2ca617d94379bdbeb3a53e19cdf0)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:38.537250 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a96540-1aad-40f9-b595-eaafb826704f admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:38.603985 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-a9a96540-1aad-40f9-b595-eaafb826704f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6c5b2ca617d94379bdbeb3a53e19cdf0'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:04:38.617178 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-a9a96540-1aad-40f9-b595-eaafb826704f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6c5b2ca617d94379bdbeb3a53e19cdf0'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:04:38.626443 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-a9a96540-1aad-40f9-b595-eaafb826704f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6c5b2ca617d94379bdbeb3a53e19cdf0'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:04:38.627153 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-a9a96540-1aad-40f9-b595-eaafb826704f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6c5b2ca617d94379bdbeb3a53e19cdf0'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:04:38.634847 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-a9a96540-1aad-40f9-b595-eaafb826704f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6c5b2ca617d94379bdbeb3a53e19cdf0'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:04:38.636419 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:38.642374 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1256/2485] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Aug 30 16:04:38 2022] DELETE /identity/v3/users/6c5b2ca617d94379bdbeb3a53e19cdf0 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:04:38.976861 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de11f00c-4df8-4be6-8527-45bea25b9861 admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:38.978343 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-de11f00c-4df8-4be6-8527-45bea25b9861 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:04:38.993035 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de11f00c-4df8-4be6-8527-45bea25b9861 admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:39.051378 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:39.063440 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bbbbae-bd97-41aa-8e91-08bbbbacf102 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:39.064212 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bbbbae-bd97-41aa-8e91-08bbbbacf102 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:39.064683 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bbbbae-bd97-41aa-8e91-08bbbbacf102 admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:39.065182 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bbbbae-bd97-41aa-8e91-08bbbbacf102 admin admin] PATH_INFO: `/v3/projects/50d8da8775684e5c8b41e65e6b1ce59a` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:39.066624 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42bbbbae-bd97-41aa-8e91-08bbbbacf102 admin admin] RBAC: Authorizing `identity:delete_project(project_id=50d8da8775684e5c8b41e65e6b1ce59a)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:39.068855 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42bbbbae-bd97-41aa-8e91-08bbbbacf102 admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:39.119850 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-42bbbbae-bd97-41aa-8e91-08bbbbacf102 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '50d8da8775684e5c8b41e65e6b1ce59a', 'request_id': 'req-42bbbbae-bd97-41aa-8e91-08bbbbacf102'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:04:39.128815 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-42bbbbae-bd97-41aa-8e91-08bbbbacf102 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '50d8da8775684e5c8b41e65e6b1ce59a', 'request_id': 'req-42bbbbae-bd97-41aa-8e91-08bbbbacf102'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:04:39.135101 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-42bbbbae-bd97-41aa-8e91-08bbbbacf102 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '50d8da8775684e5c8b41e65e6b1ce59a', 'request_id': 'req-42bbbbae-bd97-41aa-8e91-08bbbbacf102'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:04:39.145565 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-42bbbbae-bd97-41aa-8e91-08bbbbacf102 admin admin] The token cache is being invalidate because project 50d8da8775684e5c8b41e65e6b1ce59a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81512) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 16:04:39.145891 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-42bbbbae-bd97-41aa-8e91-08bbbbacf102 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:04:39.146546 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-42bbbbae-bd97-41aa-8e91-08bbbbacf102 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:04:39.148084 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1230/2486] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Aug 30 16:04:38 2022] DELETE /identity/v3/projects/50d8da8775684e5c8b41e65e6b1ce59a => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:04:39.153266 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0bec44-2963-4a0d-b11a-d40dae6a98dc None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:39.153788 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0bec44-2963-4a0d-b11a-d40dae6a98dc None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:39.153788 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0bec44-2963-4a0d-b11a-d40dae6a98dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:39.173718 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-3d0bec44-2963-4a0d-b11a-d40dae6a98dc None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:04:39.174921 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-3d0bec44-2963-4a0d-b11a-d40dae6a98dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:04:39.236556 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1257/2487] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 16:04:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:04:39.245275 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0e6c9b-5b00-469d-91c6-819cbff04f0c None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:39.245690 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0e6c9b-5b00-469d-91c6-819cbff04f0c None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:39.246130 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0e6c9b-5b00-469d-91c6-819cbff04f0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:39.276063 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-5b0e6c9b-5b00-469d-91c6-819cbff04f0c None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:04:39.277298 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-5b0e6c9b-5b00-469d-91c6-819cbff04f0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:04:39.286676 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1231/2488] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 16:04:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 16:04:39.291105 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0bec44-2963-4a0d-b11a-d40dae6a98dc None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:39.301031 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0bec44-2963-4a0d-b11a-d40dae6a98dc None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:39.306091 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba63a216-40a4-43de-aa18-feb9d67978ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:39.306692 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba63a216-40a4-43de-aa18-feb9d67978ef admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:39.306973 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba63a216-40a4-43de-aa18-feb9d67978ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:39.307245 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba63a216-40a4-43de-aa18-feb9d67978ef admin admin] PATH_INFO: `/v3/domains` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:39.307896 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba63a216-40a4-43de-aa18-feb9d67978ef admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:04:39.308194 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba63a216-40a4-43de-aa18-feb9d67978ef admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:39.309842 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba63a216-40a4-43de-aa18-feb9d67978ef admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:39.322878 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1258/2489] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Aug 30 16:04:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:04:39.328275 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0e6c9b-5b00-469d-91c6-819cbff04f0c None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:39.337654 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0e6c9b-5b00-469d-91c6-819cbff04f0c None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:39.343288 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c8438c-c0cc-4cfb-b2ba-723e2125f1c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:39.343880 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c8438c-c0cc-4cfb-b2ba-723e2125f1c5 admin admin] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:39.344157 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c8438c-c0cc-4cfb-b2ba-723e2125f1c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:39.344420 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c8438c-c0cc-4cfb-b2ba-723e2125f1c5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:39.345080 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c8438c-c0cc-4cfb-b2ba-723e2125f1c5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:39.347315 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c8438c-c0cc-4cfb-b2ba-723e2125f1c5 admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:39.390789 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1232/2490] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Aug 30 16:04:39 2022] POST /identity/v3/projects => generated 383 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:04:39.396845 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba63a216-40a4-43de-aa18-feb9d67978ef admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:39.407253 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba63a216-40a4-43de-aa18-feb9d67978ef admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:39.471498 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b9c2a1-db94-4739-9e51-d8ddae7fc8d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:39.472117 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b9c2a1-db94-4739-9e51-d8ddae7fc8d1 admin admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:39.472394 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b9c2a1-db94-4739-9e51-d8ddae7fc8d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:39.472662 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b9c2a1-db94-4739-9e51-d8ddae7fc8d1 admin admin] PATH_INFO: `/v3/users` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:39.473331 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b9c2a1-db94-4739-9e51-d8ddae7fc8d1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:39.475432 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b9c2a1-db94-4739-9e51-d8ddae7fc8d1 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:39.499307 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1259/2491] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Aug 30 16:04:39 2022] POST /identity/v3/users => generated 292 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:04:39.503584 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c8438c-c0cc-4cfb-b2ba-723e2125f1c5 admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:39.514283 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c8438c-c0cc-4cfb-b2ba-723e2125f1c5 admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:39.519827 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9670772-5be2-4565-876b-f6a8c86bdce4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:39.520415 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9670772-5be2-4565-876b-f6a8c86bdce4 admin admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:39.520669 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9670772-5be2-4565-876b-f6a8c86bdce4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:39.520950 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9670772-5be2-4565-876b-f6a8c86bdce4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:39.521622 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9670772-5be2-4565-876b-f6a8c86bdce4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:39.523231 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9670772-5be2-4565-876b-f6a8c86bdce4 admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:39.533702 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1233/2492] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 30 16:04:39 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:04:39.537482 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b9c2a1-db94-4739-9e51-d8ddae7fc8d1 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:39.547648 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b9c2a1-db94-4739-9e51-d8ddae7fc8d1 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:39.552796 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962ab5d3-a089-4c2f-97c8-774c43c9e18a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:39.553419 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962ab5d3-a089-4c2f-97c8-774c43c9e18a admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:39.553701 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962ab5d3-a089-4c2f-97c8-774c43c9e18a admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:39.553964 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962ab5d3-a089-4c2f-97c8-774c43c9e18a admin admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:39.554605 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962ab5d3-a089-4c2f-97c8-774c43c9e18a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:39.556202 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962ab5d3-a089-4c2f-97c8-774c43c9e18a admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:39.567144 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1260/2493] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 30 16:04:39 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 16:04:39.571410 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9670772-5be2-4565-876b-f6a8c86bdce4 admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:39.581775 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9670772-5be2-4565-876b-f6a8c86bdce4 admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:39.586756 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9c5f84-6493-4ae2-80fb-435c8742939a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:39.587406 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9c5f84-6493-4ae2-80fb-435c8742939a admin admin] REQUEST_METHOD: `PUT` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:39.587672 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9c5f84-6493-4ae2-80fb-435c8742939a admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:39.587943 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9c5f84-6493-4ae2-80fb-435c8742939a admin admin] PATH_INFO: `/v3/projects/6b8382e5f6db4a4fa52d905bb37107a9/users/2cc51233cd4547f58a380c2480199bf0/roles/df8f9f844b1b439181d75a07c1a93418` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:39.603186 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9c5f84-6493-4ae2-80fb-435c8742939a admin admin] RBAC: Authorizing `identity:create_grant(project_id=6b8382e5f6db4a4fa52d905bb37107a9, user_id=2cc51233cd4547f58a380c2480199bf0, role_id=df8f9f844b1b439181d75a07c1a93418)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:39.605921 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9c5f84-6493-4ae2-80fb-435c8742939a admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:39.621981 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1234/2494] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 30 16:04:39 2022] PUT /identity/v3/projects/6b8382e5f6db4a4fa52d905bb37107a9/users/2cc51233cd4547f58a380c2480199bf0/roles/df8f9f844b1b439181d75a07c1a93418 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:04:39.634141 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4612f56c-7782-4ca8-82bd-e865dc1705a7 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:39.634421 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4612f56c-7782-4ca8-82bd-e865dc1705a7 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:39.634689 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4612f56c-7782-4ca8-82bd-e865dc1705a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:39.664016 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-4612f56c-7782-4ca8-82bd-e865dc1705a7 None None] MFA Rules not processed for user `2cc51233cd4547f58a380c2480199bf0`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:04:39.665271 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-4612f56c-7782-4ca8-82bd-e865dc1705a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:04:39.722250 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:39.732917 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1261/2495] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 16:04:39 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:04:39.737987 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9c5f84-6493-4ae2-80fb-435c8742939a admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:39.799582 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9c5f84-6493-4ae2-80fb-435c8742939a admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:39.804999 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b4798e-4c6a-4821-bef9-1ccceba6230a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:39.805634 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b4798e-4c6a-4821-bef9-1ccceba6230a admin admin] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:39.805915 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b4798e-4c6a-4821-bef9-1ccceba6230a admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:39.806191 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b4798e-4c6a-4821-bef9-1ccceba6230a admin admin] PATH_INFO: `/v3/projects` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:39.806831 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b4798e-4c6a-4821-bef9-1ccceba6230a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:39.808727 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b4798e-4c6a-4821-bef9-1ccceba6230a admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:39.838601 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1235/2496] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Aug 30 16:04:39 2022] POST /identity/v3/projects => generated 381 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:04:39.842978 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4612f56c-7782-4ca8-82bd-e865dc1705a7 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:39.853728 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4612f56c-7782-4ca8-82bd-e865dc1705a7 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:39.911104 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361bb02a-b80d-4634-9ec5-ccee4af83e56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:39.911779 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361bb02a-b80d-4634-9ec5-ccee4af83e56 admin admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:39.912051 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361bb02a-b80d-4634-9ec5-ccee4af83e56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:39.912345 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361bb02a-b80d-4634-9ec5-ccee4af83e56 admin admin] PATH_INFO: `/v3/users` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:39.912990 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361bb02a-b80d-4634-9ec5-ccee4af83e56 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:39.914947 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361bb02a-b80d-4634-9ec5-ccee4af83e56 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:39.938094 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1262/2497] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Aug 30 16:04:39 2022] POST /identity/v3/users => generated 297 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 16:04:39.942381 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b4798e-4c6a-4821-bef9-1ccceba6230a admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:39.947965 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:39.954978 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b4798e-4c6a-4821-bef9-1ccceba6230a admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:39.961093 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b075c1-fdec-462b-9037-1be149884c0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:39.961761 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b075c1-fdec-462b-9037-1be149884c0e admin admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:39.962039 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b075c1-fdec-462b-9037-1be149884c0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:39.962307 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b075c1-fdec-462b-9037-1be149884c0e admin admin] PATH_INFO: `/v3/roles` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:39.962950 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b075c1-fdec-462b-9037-1be149884c0e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:39.964583 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b075c1-fdec-462b-9037-1be149884c0e admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:39.975306 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1236/2498] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 30 16:04:39 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:04:39.979697 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361bb02a-b80d-4634-9ec5-ccee4af83e56 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:39.990190 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361bb02a-b80d-4634-9ec5-ccee4af83e56 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:39.996408 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a55a40c-7fe6-445c-9454-a1756659cdbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:39.997072 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a55a40c-7fe6-445c-9454-a1756659cdbf admin admin] REQUEST_METHOD: `PUT` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:39.997387 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a55a40c-7fe6-445c-9454-a1756659cdbf admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:39.997656 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a55a40c-7fe6-445c-9454-a1756659cdbf admin admin] PATH_INFO: `/v3/projects/c8050fe8b1cf435d9bdba166752f2edc/users/7e5d9b142973405ea97141b1a8bdb51e/roles/f265386950a641ac8413b53eda4ae34d` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:40.014652 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a55a40c-7fe6-445c-9454-a1756659cdbf admin admin] RBAC: Authorizing `identity:create_grant(project_id=c8050fe8b1cf435d9bdba166752f2edc, user_id=7e5d9b142973405ea97141b1a8bdb51e, role_id=f265386950a641ac8413b53eda4ae34d)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:40.017387 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a55a40c-7fe6-445c-9454-a1756659cdbf admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:40.032716 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1263/2499] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 30 16:04:39 2022] PUT /identity/v3/projects/c8050fe8b1cf435d9bdba166752f2edc/users/7e5d9b142973405ea97141b1a8bdb51e/roles/f265386950a641ac8413b53eda4ae34d => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 16:04:40.036887 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b075c1-fdec-462b-9037-1be149884c0e admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:40.047214 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b075c1-fdec-462b-9037-1be149884c0e admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:40.104524 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0e1576-98c9-4c26-b0c7-24ca99e88edc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:40.105144 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0e1576-98c9-4c26-b0c7-24ca99e88edc admin admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:40.105537 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0e1576-98c9-4c26-b0c7-24ca99e88edc admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:40.105809 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0e1576-98c9-4c26-b0c7-24ca99e88edc admin admin] PATH_INFO: `/v3/roles` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:40.106457 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0e1576-98c9-4c26-b0c7-24ca99e88edc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:40.108415 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0e1576-98c9-4c26-b0c7-24ca99e88edc admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:40.122414 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1237/2500] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 30 16:04:40 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:04:40.123652 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a55a40c-7fe6-445c-9454-a1756659cdbf admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:40.134073 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a55a40c-7fe6-445c-9454-a1756659cdbf admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:40.138740 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b40226-7849-40e7-a016-8690959c904b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:40.139396 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b40226-7849-40e7-a016-8690959c904b admin admin] REQUEST_METHOD: `PUT` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:40.139673 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b40226-7849-40e7-a016-8690959c904b admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:40.139946 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b40226-7849-40e7-a016-8690959c904b admin admin] PATH_INFO: `/v3/projects/c8050fe8b1cf435d9bdba166752f2edc/users/7e5d9b142973405ea97141b1a8bdb51e/roles/f265386950a641ac8413b53eda4ae34d` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:40.141976 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b40226-7849-40e7-a016-8690959c904b admin admin] RBAC: Authorizing `identity:create_grant(project_id=c8050fe8b1cf435d9bdba166752f2edc, user_id=7e5d9b142973405ea97141b1a8bdb51e, role_id=f265386950a641ac8413b53eda4ae34d)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:40.144618 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b40226-7849-40e7-a016-8690959c904b admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:40.155314 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1264/2501] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 30 16:04:40 2022] PUT /identity/v3/projects/c8050fe8b1cf435d9bdba166752f2edc/users/7e5d9b142973405ea97141b1a8bdb51e/roles/f265386950a641ac8413b53eda4ae34d => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:04:40.165876 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908a9a91-43c3-47f8-b5ec-9ca354fd28d9 None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:40.166163 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908a9a91-43c3-47f8-b5ec-9ca354fd28d9 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:40.166444 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908a9a91-43c3-47f8-b5ec-9ca354fd28d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:40.194921 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-908a9a91-43c3-47f8-b5ec-9ca354fd28d9 None None] MFA Rules not processed for user `7e5d9b142973405ea97141b1a8bdb51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:04:40.196157 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-908a9a91-43c3-47f8-b5ec-9ca354fd28d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:04:40.267081 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1238/2502] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 16:04:40 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:04:40.278061 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b40226-7849-40e7-a016-8690959c904b admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:40.279855 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-d3b40226-7849-40e7-a016-8690959c904b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:04:40.336732 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b40226-7849-40e7-a016-8690959c904b admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:40.340831 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c0865e-b3ad-439c-8ac6-4cbc1979acc7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0100a04821804993866ea20f159663f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:40.341462 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c0865e-b3ad-439c-8ac6-4cbc1979acc7 service cinder] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:40.341740 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c0865e-b3ad-439c-8ac6-4cbc1979acc7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:40.342013 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c0865e-b3ad-439c-8ac6-4cbc1979acc7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:40.351823 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c0865e-b3ad-439c-8ac6-4cbc1979acc7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:40.353260 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c0865e-b3ad-439c-8ac6-4cbc1979acc7 service cinder] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:40.363250 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1265/2503] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 16:04:40 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:04:40.466826 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908a9a91-43c3-47f8-b5ec-9ca354fd28d9 None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:40.477809 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908a9a91-43c3-47f8-b5ec-9ca354fd28d9 None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:40.482450 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1342c5-4b9d-4515-b749-695640435cd9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0100a04821804993866ea20f159663f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:40.483398 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1342c5-4b9d-4515-b749-695640435cd9 service cinder] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:40.483671 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1342c5-4b9d-4515-b749-695640435cd9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:40.483952 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1342c5-4b9d-4515-b749-695640435cd9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:40.541756 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1342c5-4b9d-4515-b749-695640435cd9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:40.543168 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1342c5-4b9d-4515-b749-695640435cd9 service cinder] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:40.551896 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1239/2504] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 16:04:40 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:04:43.214586 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c0865e-b3ad-439c-8ac6-4cbc1979acc7 service cinder] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:43.267156 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:43.282225 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c0865e-b3ad-439c-8ac6-4cbc1979acc7 service cinder] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:43.286904 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c67e5ca-63ac-41d7-9852-34ba5e9f522e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:43.287528 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c67e5ca-63ac-41d7-9852-34ba5e9f522e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:43.287801 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c67e5ca-63ac-41d7-9852-34ba5e9f522e admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:43.288069 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c67e5ca-63ac-41d7-9852-34ba5e9f522e admin admin] PATH_INFO: `/v3/users/2cc51233cd4547f58a380c2480199bf0` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:43.289243 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c67e5ca-63ac-41d7-9852-34ba5e9f522e admin admin] RBAC: Authorizing `identity:delete_user(user_id=2cc51233cd4547f58a380c2480199bf0)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:43.291246 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c67e5ca-63ac-41d7-9852-34ba5e9f522e admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:43.351114 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-0c67e5ca-63ac-41d7-9852-34ba5e9f522e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2cc51233cd4547f58a380c2480199bf0'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:04:43.363703 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-0c67e5ca-63ac-41d7-9852-34ba5e9f522e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2cc51233cd4547f58a380c2480199bf0'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:04:43.374345 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-0c67e5ca-63ac-41d7-9852-34ba5e9f522e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2cc51233cd4547f58a380c2480199bf0'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:04:43.374967 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-0c67e5ca-63ac-41d7-9852-34ba5e9f522e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2cc51233cd4547f58a380c2480199bf0'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:04:43.381892 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-0c67e5ca-63ac-41d7-9852-34ba5e9f522e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2cc51233cd4547f58a380c2480199bf0'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:04:43.387752 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1266/2505] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Aug 30 16:04:43 2022] DELETE /identity/v3/users/2cc51233cd4547f58a380c2480199bf0 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:04:43.392571 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b77ddb-0a9e-4109-aab8-f1b01bb9781d None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:43.393035 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b77ddb-0a9e-4109-aab8-f1b01bb9781d None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:43.393145 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b77ddb-0a9e-4109-aab8-f1b01bb9781d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:43.413742 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-f3b77ddb-0a9e-4109-aab8-f1b01bb9781d None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:04:43.414972 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-f3b77ddb-0a9e-4109-aab8-f1b01bb9781d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:04:43.419980 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:43.477011 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1240/2506] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 16:04:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:04:43.488831 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c67e5ca-63ac-41d7-9852-34ba5e9f522e admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:43.490397 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-0c67e5ca-63ac-41d7-9852-34ba5e9f522e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:04:43.549250 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c67e5ca-63ac-41d7-9852-34ba5e9f522e admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:43.552940 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c455c7d7-3cd9-4f86-b26a-1475202d588f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f4157105f3488f807b7475008ab722', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:43.553592 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c455c7d7-3cd9-4f86-b26a-1475202d588f service neutron] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:43.553865 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c455c7d7-3cd9-4f86-b26a-1475202d588f service neutron] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:43.554131 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c455c7d7-3cd9-4f86-b26a-1475202d588f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:43.564430 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c455c7d7-3cd9-4f86-b26a-1475202d588f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:43.566727 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c455c7d7-3cd9-4f86-b26a-1475202d588f service neutron] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:43.581962 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1267/2507] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 16:04:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 16:04:43.908128 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b77ddb-0a9e-4109-aab8-f1b01bb9781d None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:43.909723 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-f3b77ddb-0a9e-4109-aab8-f1b01bb9781d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:04:43.925447 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b77ddb-0a9e-4109-aab8-f1b01bb9781d None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:43.930683 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6f9eea-aedb-405a-a77f-bb8ffe1f4486 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:43.931330 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6f9eea-aedb-405a-a77f-bb8ffe1f4486 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:43.931602 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6f9eea-aedb-405a-a77f-bb8ffe1f4486 admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:43.931864 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6f9eea-aedb-405a-a77f-bb8ffe1f4486 admin admin] PATH_INFO: `/v3/projects/6b8382e5f6db4a4fa52d905bb37107a9` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:43.933055 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6f9eea-aedb-405a-a77f-bb8ffe1f4486 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6b8382e5f6db4a4fa52d905bb37107a9)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:43.935641 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6f9eea-aedb-405a-a77f-bb8ffe1f4486 admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:43.983272 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-5a6f9eea-aedb-405a-a77f-bb8ffe1f4486 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6b8382e5f6db4a4fa52d905bb37107a9', 'request_id': 'req-5a6f9eea-aedb-405a-a77f-bb8ffe1f4486'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:04:43.996425 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-5a6f9eea-aedb-405a-a77f-bb8ffe1f4486 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6b8382e5f6db4a4fa52d905bb37107a9', 'request_id': 'req-5a6f9eea-aedb-405a-a77f-bb8ffe1f4486'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:04:44.004006 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-5a6f9eea-aedb-405a-a77f-bb8ffe1f4486 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6b8382e5f6db4a4fa52d905bb37107a9', 'request_id': 'req-5a6f9eea-aedb-405a-a77f-bb8ffe1f4486'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:04:44.015873 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-5a6f9eea-aedb-405a-a77f-bb8ffe1f4486 admin admin] The token cache is being invalidate because project 6b8382e5f6db4a4fa52d905bb37107a9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81512) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 16:04:44.016168 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-5a6f9eea-aedb-405a-a77f-bb8ffe1f4486 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:04:44.016931 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-5a6f9eea-aedb-405a-a77f-bb8ffe1f4486 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:04:44.018651 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1241/2508] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Aug 30 16:04:43 2022] DELETE /identity/v3/projects/6b8382e5f6db4a4fa52d905bb37107a9 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:04:44.022913 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c455c7d7-3cd9-4f86-b26a-1475202d588f service neutron] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:44.033174 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c455c7d7-3cd9-4f86-b26a-1475202d588f service neutron] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:44.092592 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af15f90d-5954-41b1-a613-8dd15eff542b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:44.093253 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af15f90d-5954-41b1-a613-8dd15eff542b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:44.093535 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af15f90d-5954-41b1-a613-8dd15eff542b admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:44.093813 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af15f90d-5954-41b1-a613-8dd15eff542b admin admin] PATH_INFO: `/v3/users/7e5d9b142973405ea97141b1a8bdb51e` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:44.094980 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af15f90d-5954-41b1-a613-8dd15eff542b admin admin] RBAC: Authorizing `identity:delete_user(user_id=7e5d9b142973405ea97141b1a8bdb51e)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:44.096921 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af15f90d-5954-41b1-a613-8dd15eff542b admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:44.109636 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:44.148095 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-af15f90d-5954-41b1-a613-8dd15eff542b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7e5d9b142973405ea97141b1a8bdb51e'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:04:44.161457 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-af15f90d-5954-41b1-a613-8dd15eff542b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7e5d9b142973405ea97141b1a8bdb51e'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:04:44.171106 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-af15f90d-5954-41b1-a613-8dd15eff542b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7e5d9b142973405ea97141b1a8bdb51e'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:04:44.171791 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-af15f90d-5954-41b1-a613-8dd15eff542b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7e5d9b142973405ea97141b1a8bdb51e'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:04:44.178927 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-af15f90d-5954-41b1-a613-8dd15eff542b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7e5d9b142973405ea97141b1a8bdb51e'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:04:44.185051 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1268/2509] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Aug 30 16:04:44 2022] DELETE /identity/v3/users/7e5d9b142973405ea97141b1a8bdb51e => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:04:44.475127 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6f9eea-aedb-405a-a77f-bb8ffe1f4486 admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:44.476628 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-5a6f9eea-aedb-405a-a77f-bb8ffe1f4486 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:04:44.490900 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6f9eea-aedb-405a-a77f-bb8ffe1f4486 admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:44.549898 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a75398a-0260-4d4a-9c9f-303967893355 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:44.550472 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a75398a-0260-4d4a-9c9f-303967893355 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:44.550754 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a75398a-0260-4d4a-9c9f-303967893355 admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:44.551028 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a75398a-0260-4d4a-9c9f-303967893355 admin admin] PATH_INFO: `/v3/projects/c8050fe8b1cf435d9bdba166752f2edc` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:44.552154 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a75398a-0260-4d4a-9c9f-303967893355 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c8050fe8b1cf435d9bdba166752f2edc)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:44.554123 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a75398a-0260-4d4a-9c9f-303967893355 admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:44.601664 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:44.617576 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-8a75398a-0260-4d4a-9c9f-303967893355 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c8050fe8b1cf435d9bdba166752f2edc', 'request_id': 'req-8a75398a-0260-4d4a-9c9f-303967893355'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:04:44.630743 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-8a75398a-0260-4d4a-9c9f-303967893355 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c8050fe8b1cf435d9bdba166752f2edc', 'request_id': 'req-8a75398a-0260-4d4a-9c9f-303967893355'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:04:44.638175 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-8a75398a-0260-4d4a-9c9f-303967893355 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c8050fe8b1cf435d9bdba166752f2edc', 'request_id': 'req-8a75398a-0260-4d4a-9c9f-303967893355'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:04:44.649306 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-8a75398a-0260-4d4a-9c9f-303967893355 admin admin] The token cache is being invalidate because project c8050fe8b1cf435d9bdba166752f2edc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81512) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 16:04:44.649682 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-8a75398a-0260-4d4a-9c9f-303967893355 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:04:44.650347 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-8a75398a-0260-4d4a-9c9f-303967893355 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:04:44.651845 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1242/2510] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Aug 30 16:04:44 2022] DELETE /identity/v3/projects/c8050fe8b1cf435d9bdba166752f2edc => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:04:44.656848 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8bfe0e-51a9-41a6-9baf-cc5f55c18a91 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:44.657315 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8bfe0e-51a9-41a6-9baf-cc5f55c18a91 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:44.657452 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8bfe0e-51a9-41a6-9baf-cc5f55c18a91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:44.677916 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-ad8bfe0e-51a9-41a6-9baf-cc5f55c18a91 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:04:44.679157 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-ad8bfe0e-51a9-41a6-9baf-cc5f55c18a91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:04:44.744982 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1269/2511] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 16:04:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:04:44.753107 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e91a5a-619a-482b-b4b2-0cbdd7305cb8 None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:44.753493 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e91a5a-619a-482b-b4b2-0cbdd7305cb8 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:44.753853 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e91a5a-619a-482b-b4b2-0cbdd7305cb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:44.775983 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-22e91a5a-619a-482b-b4b2-0cbdd7305cb8 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:04:44.777176 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-22e91a5a-619a-482b-b4b2-0cbdd7305cb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:04:44.785398 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1243/2512] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 16:04:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:04:44.789868 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8bfe0e-51a9-41a6-9baf-cc5f55c18a91 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:44.800268 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8bfe0e-51a9-41a6-9baf-cc5f55c18a91 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:44.806096 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09a51d1-9611-4ca7-80c0-b60a2836b445 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:44.806700 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09a51d1-9611-4ca7-80c0-b60a2836b445 admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:44.806975 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09a51d1-9611-4ca7-80c0-b60a2836b445 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:44.807238 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09a51d1-9611-4ca7-80c0-b60a2836b445 admin admin] PATH_INFO: `/v3/domains` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:44.807882 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09a51d1-9611-4ca7-80c0-b60a2836b445 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:04:44.808184 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09a51d1-9611-4ca7-80c0-b60a2836b445 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:44.809822 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09a51d1-9611-4ca7-80c0-b60a2836b445 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:44.822111 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1270/2513] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Aug 30 16:04:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:04:44.828465 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e91a5a-619a-482b-b4b2-0cbdd7305cb8 None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:44.842738 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e91a5a-619a-482b-b4b2-0cbdd7305cb8 None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:44.850995 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862af041-454b-4d92-92dc-033e12a1def7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:44.851582 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862af041-454b-4d92-92dc-033e12a1def7 admin admin] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:44.851858 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862af041-454b-4d92-92dc-033e12a1def7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:44.852118 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862af041-454b-4d92-92dc-033e12a1def7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:44.852823 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862af041-454b-4d92-92dc-033e12a1def7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:44.854743 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862af041-454b-4d92-92dc-033e12a1def7 admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:44.885126 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1244/2514] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Aug 30 16:04:44 2022] POST /identity/v3/projects => generated 371 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:04:44.889547 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09a51d1-9611-4ca7-80c0-b60a2836b445 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:44.899460 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09a51d1-9611-4ca7-80c0-b60a2836b445 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:44.960017 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4cd458-9aa1-4484-b6bd-b428ee4ec5d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:44.960630 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4cd458-9aa1-4484-b6bd-b428ee4ec5d5 admin admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:44.960930 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4cd458-9aa1-4484-b6bd-b428ee4ec5d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:44.961199 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4cd458-9aa1-4484-b6bd-b428ee4ec5d5 admin admin] PATH_INFO: `/v3/users` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:44.961888 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4cd458-9aa1-4484-b6bd-b428ee4ec5d5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:44.963821 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4cd458-9aa1-4484-b6bd-b428ee4ec5d5 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:44.991747 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1271/2515] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Aug 30 16:04:44 2022] POST /identity/v3/users => generated 286 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:04:44.993384 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862af041-454b-4d92-92dc-033e12a1def7 admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:45.004075 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862af041-454b-4d92-92dc-033e12a1def7 admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:45.010243 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d243a7e1-5b91-4ccf-bc21-c62ab0336fe1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:45.011091 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d243a7e1-5b91-4ccf-bc21-c62ab0336fe1 admin admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:45.011377 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d243a7e1-5b91-4ccf-bc21-c62ab0336fe1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:45.011644 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d243a7e1-5b91-4ccf-bc21-c62ab0336fe1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:45.012296 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d243a7e1-5b91-4ccf-bc21-c62ab0336fe1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:45.013966 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d243a7e1-5b91-4ccf-bc21-c62ab0336fe1 admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:45.033442 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1245/2516] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 30 16:04:44 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:04:45.034546 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4cd458-9aa1-4484-b6bd-b428ee4ec5d5 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:45.044535 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4cd458-9aa1-4484-b6bd-b428ee4ec5d5 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:45.049035 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6564d9c4-aeda-4768-93c3-074b85199dd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:45.049692 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6564d9c4-aeda-4768-93c3-074b85199dd1 admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:45.049955 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6564d9c4-aeda-4768-93c3-074b85199dd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:45.050228 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6564d9c4-aeda-4768-93c3-074b85199dd1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:45.050955 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6564d9c4-aeda-4768-93c3-074b85199dd1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:45.052534 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6564d9c4-aeda-4768-93c3-074b85199dd1 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:45.053926 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:45.067895 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1272/2517] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 30 16:04:45 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:04:45.069091 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d243a7e1-5b91-4ccf-bc21-c62ab0336fe1 admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:45.079098 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d243a7e1-5b91-4ccf-bc21-c62ab0336fe1 admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:45.083688 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b543bcf2-ce0c-4097-8a33-b8c765e38197 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:45.084330 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b543bcf2-ce0c-4097-8a33-b8c765e38197 admin admin] REQUEST_METHOD: `PUT` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:45.084604 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b543bcf2-ce0c-4097-8a33-b8c765e38197 admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:45.084870 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b543bcf2-ce0c-4097-8a33-b8c765e38197 admin admin] PATH_INFO: `/v3/projects/9bda495d351c4ca28612185e14019ad7/users/3b94d20427214201889a6a7cd9c0d939/roles/df8f9f844b1b439181d75a07c1a93418` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:45.099448 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b543bcf2-ce0c-4097-8a33-b8c765e38197 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9bda495d351c4ca28612185e14019ad7, user_id=3b94d20427214201889a6a7cd9c0d939, role_id=df8f9f844b1b439181d75a07c1a93418)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:45.102194 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b543bcf2-ce0c-4097-8a33-b8c765e38197 admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:45.117177 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1246/2518] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 30 16:04:45 2022] PUT /identity/v3/projects/9bda495d351c4ca28612185e14019ad7/users/3b94d20427214201889a6a7cd9c0d939/roles/df8f9f844b1b439181d75a07c1a93418 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:04:45.122667 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81b0d6a-0d07-4da5-b9ba-7035b8ba3018 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:45.122970 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81b0d6a-0d07-4da5-b9ba-7035b8ba3018 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:45.123240 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81b0d6a-0d07-4da5-b9ba-7035b8ba3018 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:45.142933 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-d81b0d6a-0d07-4da5-b9ba-7035b8ba3018 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:04:45.144149 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-d81b0d6a-0d07-4da5-b9ba-7035b8ba3018 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:04:45.214961 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1273/2519] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 16:04:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:04:45.224801 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b543bcf2-ce0c-4097-8a33-b8c765e38197 admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:45.226696 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-b543bcf2-ce0c-4097-8a33-b8c765e38197 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:04:45.284226 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b543bcf2-ce0c-4097-8a33-b8c765e38197 admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:45.288092 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbcfcde-1cf4-4bfb-84b3-8b8ab0262028 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f4157105f3488f807b7475008ab722', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:45.288694 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbcfcde-1cf4-4bfb-84b3-8b8ab0262028 service neutron] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:45.288968 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbcfcde-1cf4-4bfb-84b3-8b8ab0262028 service neutron] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:45.289245 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbcfcde-1cf4-4bfb-84b3-8b8ab0262028 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:45.298406 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbcfcde-1cf4-4bfb-84b3-8b8ab0262028 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:45.299839 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbcfcde-1cf4-4bfb-84b3-8b8ab0262028 service neutron] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:45.309497 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1247/2520] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 16:04:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:04:45.953664 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f4f8a7-d34b-4c38-8e60-09a255e4d66b None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:45.954151 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f4f8a7-d34b-4c38-8e60-09a255e4d66b None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:45.954247 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f4f8a7-d34b-4c38-8e60-09a255e4d66b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:45.983315 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-90f4f8a7-d34b-4c38-8e60-09a255e4d66b None None] MFA Rules not processed for user `3b94d20427214201889a6a7cd9c0d939`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:04:45.984607 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-90f4f8a7-d34b-4c38-8e60-09a255e4d66b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:04:46.064034 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1274/2521] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 16:04:45 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:04:46.065837 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbcfcde-1cf4-4bfb-84b3-8b8ab0262028 service neutron] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:46.078967 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbcfcde-1cf4-4bfb-84b3-8b8ab0262028 service neutron] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:46.084569 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccead9c-8c84-4e58-a728-3078bd11494a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:46.085540 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccead9c-8c84-4e58-a728-3078bd11494a admin admin] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:46.085540 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccead9c-8c84-4e58-a728-3078bd11494a admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:46.085744 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccead9c-8c84-4e58-a728-3078bd11494a admin admin] PATH_INFO: `/v3/projects` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:46.086378 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccead9c-8c84-4e58-a728-3078bd11494a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:46.089249 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccead9c-8c84-4e58-a728-3078bd11494a admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:46.126698 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1248/2522] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Aug 30 16:04:46 2022] POST /identity/v3/projects => generated 373 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:04:46.127731 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f4f8a7-d34b-4c38-8e60-09a255e4d66b None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:46.140799 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f4f8a7-d34b-4c38-8e60-09a255e4d66b None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:46.209983 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba95c8e-817f-4ec9-b8b4-b09eed88a70a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:46.210590 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba95c8e-817f-4ec9-b8b4-b09eed88a70a admin admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:46.210863 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba95c8e-817f-4ec9-b8b4-b09eed88a70a admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:46.211126 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba95c8e-817f-4ec9-b8b4-b09eed88a70a admin admin] PATH_INFO: `/v3/users` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:46.211783 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba95c8e-817f-4ec9-b8b4-b09eed88a70a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:46.213737 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba95c8e-817f-4ec9-b8b4-b09eed88a70a admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:46.236649 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1275/2523] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Aug 30 16:04:46 2022] POST /identity/v3/users => generated 293 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:04:46.240729 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccead9c-8c84-4e58-a728-3078bd11494a admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:46.251775 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccead9c-8c84-4e58-a728-3078bd11494a admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:46.256398 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2921997-e513-4955-9727-2bb49a680919 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:46.256994 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2921997-e513-4955-9727-2bb49a680919 admin admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:46.257369 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2921997-e513-4955-9727-2bb49a680919 admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:46.257629 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2921997-e513-4955-9727-2bb49a680919 admin admin] PATH_INFO: `/v3/roles` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:46.258270 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2921997-e513-4955-9727-2bb49a680919 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:46.260002 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2921997-e513-4955-9727-2bb49a680919 admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:46.270685 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1249/2524] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 30 16:04:46 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:04:46.275026 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba95c8e-817f-4ec9-b8b4-b09eed88a70a admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:46.285608 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba95c8e-817f-4ec9-b8b4-b09eed88a70a admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:46.291088 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4082dc20-0eac-4dc0-b3d2-2ff0b9761379 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:46.291739 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4082dc20-0eac-4dc0-b3d2-2ff0b9761379 admin admin] REQUEST_METHOD: `PUT` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:46.292019 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4082dc20-0eac-4dc0-b3d2-2ff0b9761379 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:46.292290 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4082dc20-0eac-4dc0-b3d2-2ff0b9761379 admin admin] PATH_INFO: `/v3/projects/dc63549897184860bd1675dafc0ac67a/users/45062d18d368433e954872bbc5e5824e/roles/f265386950a641ac8413b53eda4ae34d` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:46.304295 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:46.307698 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4082dc20-0eac-4dc0-b3d2-2ff0b9761379 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc63549897184860bd1675dafc0ac67a, user_id=45062d18d368433e954872bbc5e5824e, role_id=f265386950a641ac8413b53eda4ae34d)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:46.310614 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4082dc20-0eac-4dc0-b3d2-2ff0b9761379 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:46.326623 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1276/2525] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 30 16:04:46 2022] PUT /identity/v3/projects/dc63549897184860bd1675dafc0ac67a/users/45062d18d368433e954872bbc5e5824e/roles/f265386950a641ac8413b53eda4ae34d => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:04:46.330699 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2921997-e513-4955-9727-2bb49a680919 admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:46.340868 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2921997-e513-4955-9727-2bb49a680919 admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:46.370472 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:46.402580 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838181c6-367b-4aa3-a550-82d0ed5b0340 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:46.403105 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838181c6-367b-4aa3-a550-82d0ed5b0340 admin admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:46.403405 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838181c6-367b-4aa3-a550-82d0ed5b0340 admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:46.403653 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838181c6-367b-4aa3-a550-82d0ed5b0340 admin admin] PATH_INFO: `/v3/roles` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:46.404359 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838181c6-367b-4aa3-a550-82d0ed5b0340 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:46.406016 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838181c6-367b-4aa3-a550-82d0ed5b0340 admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:46.415792 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1250/2526] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 30 16:04:46 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:04:46.420213 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4082dc20-0eac-4dc0-b3d2-2ff0b9761379 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:46.431138 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4082dc20-0eac-4dc0-b3d2-2ff0b9761379 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:46.436425 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c386998d-3520-46c0-9982-ddb96b4985e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:46.437105 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c386998d-3520-46c0-9982-ddb96b4985e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:46.437421 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c386998d-3520-46c0-9982-ddb96b4985e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:46.437687 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c386998d-3520-46c0-9982-ddb96b4985e9 admin admin] PATH_INFO: `/v3/projects/dc63549897184860bd1675dafc0ac67a/users/45062d18d368433e954872bbc5e5824e/roles/f265386950a641ac8413b53eda4ae34d` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:46.439628 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c386998d-3520-46c0-9982-ddb96b4985e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc63549897184860bd1675dafc0ac67a, user_id=45062d18d368433e954872bbc5e5824e, role_id=f265386950a641ac8413b53eda4ae34d)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:46.442353 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c386998d-3520-46c0-9982-ddb96b4985e9 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:46.453125 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1277/2527] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 30 16:04:46 2022] PUT /identity/v3/projects/dc63549897184860bd1675dafc0ac67a/users/45062d18d368433e954872bbc5e5824e/roles/f265386950a641ac8413b53eda4ae34d => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:04:47.124620 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5a6847-9209-4486-90fc-1b056e6b1996 None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:47.125164 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5a6847-9209-4486-90fc-1b056e6b1996 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:47.125164 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5a6847-9209-4486-90fc-1b056e6b1996 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:47.155115 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-aa5a6847-9209-4486-90fc-1b056e6b1996 None None] MFA Rules not processed for user `45062d18d368433e954872bbc5e5824e`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:04:47.156586 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-aa5a6847-9209-4486-90fc-1b056e6b1996 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:04:47.242872 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1251/2528] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 16:04:47 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:04:47.256571 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c386998d-3520-46c0-9982-ddb96b4985e9 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:47.259063 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-c386998d-3520-46c0-9982-ddb96b4985e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:04:47.324195 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c386998d-3520-46c0-9982-ddb96b4985e9 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:47.327702 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b14b7f7-8005-4711-9708-6668c78c57f2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0100a04821804993866ea20f159663f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:47.328606 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b14b7f7-8005-4711-9708-6668c78c57f2 service cinder] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:47.328980 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b14b7f7-8005-4711-9708-6668c78c57f2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:47.329380 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b14b7f7-8005-4711-9708-6668c78c57f2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:47.392449 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b14b7f7-8005-4711-9708-6668c78c57f2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:47.393996 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b14b7f7-8005-4711-9708-6668c78c57f2 service cinder] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:47.403147 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1278/2529] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 16:04:47 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:04:48.769036 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5a6847-9209-4486-90fc-1b056e6b1996 None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:48.770765 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-aa5a6847-9209-4486-90fc-1b056e6b1996 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:04:48.784963 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5a6847-9209-4486-90fc-1b056e6b1996 None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:48.791183 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25b45eb-e76b-4c6c-af3e-5cbae873a7e7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0100a04821804993866ea20f159663f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:48.791810 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25b45eb-e76b-4c6c-af3e-5cbae873a7e7 service cinder] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:48.792084 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25b45eb-e76b-4c6c-af3e-5cbae873a7e7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:48.792350 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25b45eb-e76b-4c6c-af3e-5cbae873a7e7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:48.802118 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25b45eb-e76b-4c6c-af3e-5cbae873a7e7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:48.803582 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25b45eb-e76b-4c6c-af3e-5cbae873a7e7 service cinder] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:48.813414 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1252/2530] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 16:04:48 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:04:57.294610 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20129ef-6eac-4107-b605-f441af70dda4 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:57.295098 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20129ef-6eac-4107-b605-f441af70dda4 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:57.295194 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20129ef-6eac-4107-b605-f441af70dda4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:57.314846 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-c20129ef-6eac-4107-b605-f441af70dda4 None None] MFA Rules not processed for user `07130d598131443486c2bc1c6673c45b`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:04:57.316078 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-c20129ef-6eac-4107-b605-f441af70dda4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:04:57.375617 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1279/2531] 10.222.0.20 () {58 vars in 1009 bytes} [Tue Aug 30 16:04:57 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:04:57.424870 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25b45eb-e76b-4c6c-af3e-5cbae873a7e7 service cinder] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:57.426342 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-a25b45eb-e76b-4c6c-af3e-5cbae873a7e7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:04:57.504993 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25b45eb-e76b-4c6c-af3e-5cbae873a7e7 service cinder] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:57.511331 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56559fc8-dd13-44b8-bf29-9c02ebd91913 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8fdafdc39a594bb4b0aeaf2fbc84c2fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:57.511930 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56559fc8-dd13-44b8-bf29-9c02ebd91913 service swift] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:57.512171 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56559fc8-dd13-44b8-bf29-9c02ebd91913 service swift] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:57.512466 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56559fc8-dd13-44b8-bf29-9c02ebd91913 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:57.520668 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56559fc8-dd13-44b8-bf29-9c02ebd91913 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:57.522301 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56559fc8-dd13-44b8-bf29-9c02ebd91913 service swift] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:57.530007 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1253/2532] 10.222.0.20 () {60 vars in 1384 bytes} [Tue Aug 30 16:04:57 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 30 16:04:59.092051 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20129ef-6eac-4107-b605-f441af70dda4 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:59.094481 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-c20129ef-6eac-4107-b605-f441af70dda4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:04:59.167915 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:59.236436 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20129ef-6eac-4107-b605-f441af70dda4 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:59.243215 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c18a87a-a565-49bd-8bc7-b1257137bfd3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a1e756ae9be4a7bac1e81acf07e8f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:59.243929 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c18a87a-a565-49bd-8bc7-b1257137bfd3 service nova] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:59.244212 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c18a87a-a565-49bd-8bc7-b1257137bfd3 service nova] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:59.244484 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c18a87a-a565-49bd-8bc7-b1257137bfd3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:59.246345 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-3c18a87a-a565-49bd-8bc7-b1257137bfd3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:04:59.262583 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c18a87a-a565-49bd-8bc7-b1257137bfd3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:59.264372 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c18a87a-a565-49bd-8bc7-b1257137bfd3 service nova] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:59.278601 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1280/2533] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 16:04:59 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:05:02.416697 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56559fc8-dd13-44b8-bf29-9c02ebd91913 service swift] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:02.466597 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:02.508571 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56559fc8-dd13-44b8-bf29-9c02ebd91913 service swift] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:02.514946 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91eab6f9-c20b-4102-9f20-9bc02c29b172 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:02.515585 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91eab6f9-c20b-4102-9f20-9bc02c29b172 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:02.515864 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91eab6f9-c20b-4102-9f20-9bc02c29b172 admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:02.516132 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91eab6f9-c20b-4102-9f20-9bc02c29b172 admin admin] PATH_INFO: `/v3/users/3b94d20427214201889a6a7cd9c0d939` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:02.517296 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91eab6f9-c20b-4102-9f20-9bc02c29b172 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3b94d20427214201889a6a7cd9c0d939)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:02.519228 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91eab6f9-c20b-4102-9f20-9bc02c29b172 admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:02.583110 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-91eab6f9-c20b-4102-9f20-9bc02c29b172 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3b94d20427214201889a6a7cd9c0d939'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:05:02.595297 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-91eab6f9-c20b-4102-9f20-9bc02c29b172 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3b94d20427214201889a6a7cd9c0d939'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:05:02.614805 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-91eab6f9-c20b-4102-9f20-9bc02c29b172 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3b94d20427214201889a6a7cd9c0d939'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:05:02.624565 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-91eab6f9-c20b-4102-9f20-9bc02c29b172 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3b94d20427214201889a6a7cd9c0d939'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:05:02.625662 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-91eab6f9-c20b-4102-9f20-9bc02c29b172 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3b94d20427214201889a6a7cd9c0d939'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:05:02.632799 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1254/2534] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Aug 30 16:05:02 2022] DELETE /identity/v3/users/3b94d20427214201889a6a7cd9c0d939 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:05:02.863544 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c18a87a-a565-49bd-8bc7-b1257137bfd3 service nova] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:02.878899 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c18a87a-a565-49bd-8bc7-b1257137bfd3 service nova] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:02.957281 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27bd5ce-35c7-40c6-9c5b-c8368b8ce238 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:02.957281 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27bd5ce-35c7-40c6-9c5b-c8368b8ce238 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:02.957281 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27bd5ce-35c7-40c6-9c5b-c8368b8ce238 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:02.957281 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27bd5ce-35c7-40c6-9c5b-c8368b8ce238 admin admin] PATH_INFO: `/v3/projects/9bda495d351c4ca28612185e14019ad7` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:02.958244 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27bd5ce-35c7-40c6-9c5b-c8368b8ce238 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9bda495d351c4ca28612185e14019ad7)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:02.959346 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27bd5ce-35c7-40c6-9c5b-c8368b8ce238 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:03.037333 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-f27bd5ce-35c7-40c6-9c5b-c8368b8ce238 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9bda495d351c4ca28612185e14019ad7', 'request_id': 'req-f27bd5ce-35c7-40c6-9c5b-c8368b8ce238'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:05:03.040412 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-f27bd5ce-35c7-40c6-9c5b-c8368b8ce238 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9bda495d351c4ca28612185e14019ad7', 'request_id': 'req-f27bd5ce-35c7-40c6-9c5b-c8368b8ce238'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:05:03.051662 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-f27bd5ce-35c7-40c6-9c5b-c8368b8ce238 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9bda495d351c4ca28612185e14019ad7', 'request_id': 'req-f27bd5ce-35c7-40c6-9c5b-c8368b8ce238'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:05:03.070772 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-f27bd5ce-35c7-40c6-9c5b-c8368b8ce238 admin admin] The token cache is being invalidate because project 9bda495d351c4ca28612185e14019ad7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81511) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 16:05:03.080014 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-f27bd5ce-35c7-40c6-9c5b-c8368b8ce238 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:05:03.080014 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-f27bd5ce-35c7-40c6-9c5b-c8368b8ce238 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:05:03.080014 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1281/2535] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Aug 30 16:05:02 2022] DELETE /identity/v3/projects/9bda495d351c4ca28612185e14019ad7 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:05:03.083202 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91eab6f9-c20b-4102-9f20-9bc02c29b172 admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:03.087219 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-91eab6f9-c20b-4102-9f20-9bc02c29b172 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:05:03.100469 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91eab6f9-c20b-4102-9f20-9bc02c29b172 admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:03.168771 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73cd5fb-2ff0-49d9-bf63-1ce90db8dad3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:03.169482 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73cd5fb-2ff0-49d9-bf63-1ce90db8dad3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:03.169740 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73cd5fb-2ff0-49d9-bf63-1ce90db8dad3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:03.170223 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73cd5fb-2ff0-49d9-bf63-1ce90db8dad3 admin admin] PATH_INFO: `/v3/users/45062d18d368433e954872bbc5e5824e` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:03.171848 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73cd5fb-2ff0-49d9-bf63-1ce90db8dad3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=45062d18d368433e954872bbc5e5824e)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:03.175154 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73cd5fb-2ff0-49d9-bf63-1ce90db8dad3 admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:03.230236 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-c73cd5fb-2ff0-49d9-bf63-1ce90db8dad3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '45062d18d368433e954872bbc5e5824e'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:05:03.233686 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:03.242914 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-c73cd5fb-2ff0-49d9-bf63-1ce90db8dad3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '45062d18d368433e954872bbc5e5824e'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:05:03.260702 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-c73cd5fb-2ff0-49d9-bf63-1ce90db8dad3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '45062d18d368433e954872bbc5e5824e'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:05:03.269207 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-c73cd5fb-2ff0-49d9-bf63-1ce90db8dad3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '45062d18d368433e954872bbc5e5824e'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:05:03.269977 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-c73cd5fb-2ff0-49d9-bf63-1ce90db8dad3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '45062d18d368433e954872bbc5e5824e'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:05:03.277071 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1255/2536] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Aug 30 16:05:03 2022] DELETE /identity/v3/users/45062d18d368433e954872bbc5e5824e => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:05:03.518928 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27bd5ce-35c7-40c6-9c5b-c8368b8ce238 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:03.520561 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-f27bd5ce-35c7-40c6-9c5b-c8368b8ce238 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:05:03.544554 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27bd5ce-35c7-40c6-9c5b-c8368b8ce238 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:03.668966 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:03.685364 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7f6f65-4397-446a-a9fa-ba6445c1b301 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:03.686944 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7f6f65-4397-446a-a9fa-ba6445c1b301 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:03.687627 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7f6f65-4397-446a-a9fa-ba6445c1b301 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:03.688816 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7f6f65-4397-446a-a9fa-ba6445c1b301 admin admin] PATH_INFO: `/v3/projects/dc63549897184860bd1675dafc0ac67a` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:03.690461 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7f6f65-4397-446a-a9fa-ba6445c1b301 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dc63549897184860bd1675dafc0ac67a)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:03.693298 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7f6f65-4397-446a-a9fa-ba6445c1b301 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:03.766396 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-6a7f6f65-4397-446a-a9fa-ba6445c1b301 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dc63549897184860bd1675dafc0ac67a', 'request_id': 'req-6a7f6f65-4397-446a-a9fa-ba6445c1b301'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:05:03.782838 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-6a7f6f65-4397-446a-a9fa-ba6445c1b301 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dc63549897184860bd1675dafc0ac67a', 'request_id': 'req-6a7f6f65-4397-446a-a9fa-ba6445c1b301'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:05:03.790251 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-6a7f6f65-4397-446a-a9fa-ba6445c1b301 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dc63549897184860bd1675dafc0ac67a', 'request_id': 'req-6a7f6f65-4397-446a-a9fa-ba6445c1b301'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:05:03.815487 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-6a7f6f65-4397-446a-a9fa-ba6445c1b301 admin admin] The token cache is being invalidate because project dc63549897184860bd1675dafc0ac67a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81511) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 16:05:03.815844 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-6a7f6f65-4397-446a-a9fa-ba6445c1b301 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:05:03.816437 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-6a7f6f65-4397-446a-a9fa-ba6445c1b301 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:05:03.826916 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1282/2537] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Aug 30 16:05:03 2022] DELETE /identity/v3/projects/dc63549897184860bd1675dafc0ac67a => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:05:03.831145 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1667aeef-a3f8-4d32-9800-eb0b0d58d0c8 None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:03.831661 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1667aeef-a3f8-4d32-9800-eb0b0d58d0c8 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:03.831782 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1667aeef-a3f8-4d32-9800-eb0b0d58d0c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:03.853689 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-1667aeef-a3f8-4d32-9800-eb0b0d58d0c8 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:05:03.854968 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-1667aeef-a3f8-4d32-9800-eb0b0d58d0c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:05:03.939765 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1256/2538] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 16:05:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:05:03.949267 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7155fff8-0f49-4dea-b99d-0f8bf62c9160 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:03.949830 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7155fff8-0f49-4dea-b99d-0f8bf62c9160 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:03.949830 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7155fff8-0f49-4dea-b99d-0f8bf62c9160 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:03.974706 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-7155fff8-0f49-4dea-b99d-0f8bf62c9160 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:05:03.976039 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-7155fff8-0f49-4dea-b99d-0f8bf62c9160 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:05:03.985826 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1283/2539] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 16:05:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:05:03.990077 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1667aeef-a3f8-4d32-9800-eb0b0d58d0c8 None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:04.000855 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1667aeef-a3f8-4d32-9800-eb0b0d58d0c8 None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:04.005737 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cdf70b-37c2-492f-a194-87fffd514d6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:04.006410 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cdf70b-37c2-492f-a194-87fffd514d6c admin admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:04.006810 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cdf70b-37c2-492f-a194-87fffd514d6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:04.007078 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cdf70b-37c2-492f-a194-87fffd514d6c admin admin] PATH_INFO: `/v3/domains` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:04.007732 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4cdf70b-37c2-492f-a194-87fffd514d6c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81512) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:05:04.008038 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4cdf70b-37c2-492f-a194-87fffd514d6c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:04.009672 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4cdf70b-37c2-492f-a194-87fffd514d6c admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:04.025344 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1257/2540] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Aug 30 16:05:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:05:04.029518 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7155fff8-0f49-4dea-b99d-0f8bf62c9160 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:04.042352 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7155fff8-0f49-4dea-b99d-0f8bf62c9160 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:04.047301 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1321fe53-03a1-44d4-951b-9da91a60040e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:04.047914 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1321fe53-03a1-44d4-951b-9da91a60040e admin admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:04.048190 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1321fe53-03a1-44d4-951b-9da91a60040e admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:04.049463 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1321fe53-03a1-44d4-951b-9da91a60040e admin admin] PATH_INFO: `/v3/projects` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:04.049463 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1321fe53-03a1-44d4-951b-9da91a60040e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:04.051330 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1321fe53-03a1-44d4-951b-9da91a60040e admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:04.086400 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1284/2541] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Aug 30 16:05:04 2022] POST /identity/v3/projects => generated 385 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:05:04.092267 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cdf70b-37c2-492f-a194-87fffd514d6c admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:04.105712 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cdf70b-37c2-492f-a194-87fffd514d6c admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:04.181848 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2987dc-cbfb-45f5-9740-dfc7141c4723 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:04.182632 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2987dc-cbfb-45f5-9740-dfc7141c4723 admin admin] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:04.183122 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2987dc-cbfb-45f5-9740-dfc7141c4723 admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:04.183493 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2987dc-cbfb-45f5-9740-dfc7141c4723 admin admin] PATH_INFO: `/v3/users` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:04.184412 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2987dc-cbfb-45f5-9740-dfc7141c4723 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:04.186502 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2987dc-cbfb-45f5-9740-dfc7141c4723 admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:04.213130 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1258/2542] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Aug 30 16:05:04 2022] POST /identity/v3/users => generated 293 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 16:05:04.217333 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1321fe53-03a1-44d4-951b-9da91a60040e admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:04.235310 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1321fe53-03a1-44d4-951b-9da91a60040e admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:04.242068 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4758dc26-4aff-4348-afdc-0d8d0ee388c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:04.242068 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4758dc26-4aff-4348-afdc-0d8d0ee388c0 admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:04.242068 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4758dc26-4aff-4348-afdc-0d8d0ee388c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:04.242068 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4758dc26-4aff-4348-afdc-0d8d0ee388c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:04.242568 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4758dc26-4aff-4348-afdc-0d8d0ee388c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:04.244249 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4758dc26-4aff-4348-afdc-0d8d0ee388c0 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:04.261548 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1285/2543] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 30 16:05:04 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:05:04.265749 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2987dc-cbfb-45f5-9740-dfc7141c4723 admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:04.276600 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2987dc-cbfb-45f5-9740-dfc7141c4723 admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:04.283510 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5dbc17-75a9-4a70-9d9a-74b5829e219a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:04.284401 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5dbc17-75a9-4a70-9d9a-74b5829e219a admin admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:04.284781 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5dbc17-75a9-4a70-9d9a-74b5829e219a admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:04.285149 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5dbc17-75a9-4a70-9d9a-74b5829e219a admin admin] PATH_INFO: `/v3/roles` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:04.285955 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5dbc17-75a9-4a70-9d9a-74b5829e219a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:04.287837 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5dbc17-75a9-4a70-9d9a-74b5829e219a admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:04.301049 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1259/2544] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 30 16:05:04 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:05:04.305415 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4758dc26-4aff-4348-afdc-0d8d0ee388c0 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:04.316085 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4758dc26-4aff-4348-afdc-0d8d0ee388c0 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:04.321514 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a8b863-7e29-4d9c-916d-1cdd1a0b0534 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:04.322149 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a8b863-7e29-4d9c-916d-1cdd1a0b0534 admin admin] REQUEST_METHOD: `PUT` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:04.322423 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a8b863-7e29-4d9c-916d-1cdd1a0b0534 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:04.322692 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a8b863-7e29-4d9c-916d-1cdd1a0b0534 admin admin] PATH_INFO: `/v3/projects/a5457c7c1d774af4bb9a931a72091858/users/e159d449db00454a802e5c4437a37f80/roles/df8f9f844b1b439181d75a07c1a93418` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:04.351738 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a8b863-7e29-4d9c-916d-1cdd1a0b0534 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a5457c7c1d774af4bb9a931a72091858, user_id=e159d449db00454a802e5c4437a37f80, role_id=df8f9f844b1b439181d75a07c1a93418)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:04.354491 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a8b863-7e29-4d9c-916d-1cdd1a0b0534 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:04.384535 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1286/2545] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 30 16:05:04 2022] PUT /identity/v3/projects/a5457c7c1d774af4bb9a931a72091858/users/e159d449db00454a802e5c4437a37f80/roles/df8f9f844b1b439181d75a07c1a93418 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:05:04.386640 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129bca2f-4e5c-49ec-87aa-77c496fad4e1 None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:04.386933 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129bca2f-4e5c-49ec-87aa-77c496fad4e1 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:04.387208 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129bca2f-4e5c-49ec-87aa-77c496fad4e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:04.419089 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-129bca2f-4e5c-49ec-87aa-77c496fad4e1 None None] MFA Rules not processed for user `e159d449db00454a802e5c4437a37f80`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:05:04.420363 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-129bca2f-4e5c-49ec-87aa-77c496fad4e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:05:04.444082 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:DELETE=3,keystone:SELECT=94,keystone:INSERT=3 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:04.500979 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1260/2546] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 16:05:04 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:05:04.502164 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a8b863-7e29-4d9c-916d-1cdd1a0b0534 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:04.571011 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a8b863-7e29-4d9c-916d-1cdd1a0b0534 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:04.576623 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7b2e10-0b70-449a-9990-bc5032ede859 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:04.577280 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7b2e10-0b70-449a-9990-bc5032ede859 admin admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:04.577684 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7b2e10-0b70-449a-9990-bc5032ede859 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:04.577963 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7b2e10-0b70-449a-9990-bc5032ede859 admin admin] PATH_INFO: `/v3/projects` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:04.578609 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7b2e10-0b70-449a-9990-bc5032ede859 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:04.581797 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7b2e10-0b70-449a-9990-bc5032ede859 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:04.623048 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1287/2547] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Aug 30 16:05:04 2022] POST /identity/v3/projects => generated 383 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:05:04.626735 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129bca2f-4e5c-49ec-87aa-77c496fad4e1 None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:04.646827 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129bca2f-4e5c-49ec-87aa-77c496fad4e1 None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:04.725718 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89e909a-bab2-493e-99b0-51d20fa9477d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:04.726342 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89e909a-bab2-493e-99b0-51d20fa9477d admin admin] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:04.726619 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89e909a-bab2-493e-99b0-51d20fa9477d admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:04.726892 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89e909a-bab2-493e-99b0-51d20fa9477d admin admin] PATH_INFO: `/v3/users` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:04.727549 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89e909a-bab2-493e-99b0-51d20fa9477d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:04.729541 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89e909a-bab2-493e-99b0-51d20fa9477d admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:04.760595 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1261/2548] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Aug 30 16:05:04 2022] POST /identity/v3/users => generated 292 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:05:04.765618 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7b2e10-0b70-449a-9990-bc5032ede859 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:04.772913 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:04.781755 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7b2e10-0b70-449a-9990-bc5032ede859 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:04.789552 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9450cf03-ec9d-4119-918e-342cd6851517 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:04.790417 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9450cf03-ec9d-4119-918e-342cd6851517 admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:04.790866 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9450cf03-ec9d-4119-918e-342cd6851517 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:04.791238 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9450cf03-ec9d-4119-918e-342cd6851517 admin admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:04.791992 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9450cf03-ec9d-4119-918e-342cd6851517 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:04.804407 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9450cf03-ec9d-4119-918e-342cd6851517 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:04.828796 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1288/2549] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 30 16:05:04 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 16:05:04.833292 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89e909a-bab2-493e-99b0-51d20fa9477d admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:04.844593 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89e909a-bab2-493e-99b0-51d20fa9477d admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:04.851701 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacb1a61-81b9-45f4-8d62-5c91fdbc7dcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:04.852333 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacb1a61-81b9-45f4-8d62-5c91fdbc7dcc admin admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:04.852611 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacb1a61-81b9-45f4-8d62-5c91fdbc7dcc admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:04.852882 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacb1a61-81b9-45f4-8d62-5c91fdbc7dcc admin admin] PATH_INFO: `/v3/roles` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:04.853674 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dacb1a61-81b9-45f4-8d62-5c91fdbc7dcc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:04.855342 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dacb1a61-81b9-45f4-8d62-5c91fdbc7dcc admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:04.878889 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1262/2550] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 30 16:05:04 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 16:05:04.882823 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9450cf03-ec9d-4119-918e-342cd6851517 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:04.895762 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9450cf03-ec9d-4119-918e-342cd6851517 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:04.900874 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010ca03a-aa29-4484-9ef0-2823d55d3ac5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:04.901582 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010ca03a-aa29-4484-9ef0-2823d55d3ac5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:04.901859 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010ca03a-aa29-4484-9ef0-2823d55d3ac5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:04.902130 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010ca03a-aa29-4484-9ef0-2823d55d3ac5 admin admin] PATH_INFO: `/v3/projects/d8e35fde3e29488e82dd5ac977cc2bd8/users/1950c274d93b480bad1e8ef0d2438b4e/roles/df8f9f844b1b439181d75a07c1a93418` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:04.918523 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010ca03a-aa29-4484-9ef0-2823d55d3ac5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d8e35fde3e29488e82dd5ac977cc2bd8, user_id=1950c274d93b480bad1e8ef0d2438b4e, role_id=df8f9f844b1b439181d75a07c1a93418)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:04.921380 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010ca03a-aa29-4484-9ef0-2823d55d3ac5 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:04.945153 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1289/2551] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 30 16:05:04 2022] PUT /identity/v3/projects/d8e35fde3e29488e82dd5ac977cc2bd8/users/1950c274d93b480bad1e8ef0d2438b4e/roles/df8f9f844b1b439181d75a07c1a93418 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:05:04.956101 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4572cf68-b4da-4448-b54c-b0d1bc9a6b6b None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:04.956413 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4572cf68-b4da-4448-b54c-b0d1bc9a6b6b None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:04.956689 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4572cf68-b4da-4448-b54c-b0d1bc9a6b6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:05.012454 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-4572cf68-b4da-4448-b54c-b0d1bc9a6b6b None None] MFA Rules not processed for user `1950c274d93b480bad1e8ef0d2438b4e`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:05:05.014173 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-4572cf68-b4da-4448-b54c-b0d1bc9a6b6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:05:05.101343 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1263/2552] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 16:05:04 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:05:05.104680 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010ca03a-aa29-4484-9ef0-2823d55d3ac5 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:05.181862 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010ca03a-aa29-4484-9ef0-2823d55d3ac5 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:05.187203 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f334912-c900-41b8-9887-d0e8dda1c029 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:05.187910 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f334912-c900-41b8-9887-d0e8dda1c029 admin admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:05.188282 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f334912-c900-41b8-9887-d0e8dda1c029 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:05.188713 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f334912-c900-41b8-9887-d0e8dda1c029 admin admin] PATH_INFO: `/v3/projects` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:05.189399 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f334912-c900-41b8-9887-d0e8dda1c029 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:05.191421 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f334912-c900-41b8-9887-d0e8dda1c029 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:05.225253 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1290/2553] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Aug 30 16:05:05 2022] POST /identity/v3/projects => generated 385 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:05:05.228349 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4572cf68-b4da-4448-b54c-b0d1bc9a6b6b None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:05.240668 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4572cf68-b4da-4448-b54c-b0d1bc9a6b6b None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:05.287117 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:05.322894 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d408c71-264c-4c5f-bbd4-c754511903de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:05.323683 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d408c71-264c-4c5f-bbd4-c754511903de admin admin] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:05.324060 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d408c71-264c-4c5f-bbd4-c754511903de admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:05.324436 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d408c71-264c-4c5f-bbd4-c754511903de admin admin] PATH_INFO: `/v3/users` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:05.325265 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d408c71-264c-4c5f-bbd4-c754511903de admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:05.327325 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d408c71-264c-4c5f-bbd4-c754511903de admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:05.360710 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1264/2554] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Aug 30 16:05:05 2022] POST /identity/v3/users => generated 299 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:05:05.361933 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f334912-c900-41b8-9887-d0e8dda1c029 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:05.375642 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f334912-c900-41b8-9887-d0e8dda1c029 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:05.385689 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349c518e-1293-47f1-b38e-b7c01b071146 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:05.386370 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349c518e-1293-47f1-b38e-b7c01b071146 admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:05.386659 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349c518e-1293-47f1-b38e-b7c01b071146 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:05.386936 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349c518e-1293-47f1-b38e-b7c01b071146 admin admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:05.390679 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349c518e-1293-47f1-b38e-b7c01b071146 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:05.393116 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349c518e-1293-47f1-b38e-b7c01b071146 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:05.410359 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1291/2555] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 30 16:05:05 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:05:05.412056 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d408c71-264c-4c5f-bbd4-c754511903de admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:05.423316 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d408c71-264c-4c5f-bbd4-c754511903de admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:05.428154 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4712aa-5d41-4744-a2cc-1d7db825887c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:05.428981 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4712aa-5d41-4744-a2cc-1d7db825887c admin admin] REQUEST_METHOD: `PUT` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:05.429297 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4712aa-5d41-4744-a2cc-1d7db825887c admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:05.429579 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4712aa-5d41-4744-a2cc-1d7db825887c admin admin] PATH_INFO: `/v3/projects/d6bbd8936e9645209e7af0e9dbe4e12a/users/c531981406634c20b211c3081034c036/roles/f265386950a641ac8413b53eda4ae34d` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:05.453692 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4712aa-5d41-4744-a2cc-1d7db825887c admin admin] RBAC: Authorizing `identity:create_grant(project_id=d6bbd8936e9645209e7af0e9dbe4e12a, user_id=c531981406634c20b211c3081034c036, role_id=f265386950a641ac8413b53eda4ae34d)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:05.456715 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4712aa-5d41-4744-a2cc-1d7db825887c admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:05.477657 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1265/2556] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 30 16:05:05 2022] PUT /identity/v3/projects/d6bbd8936e9645209e7af0e9dbe4e12a/users/c531981406634c20b211c3081034c036/roles/f265386950a641ac8413b53eda4ae34d => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 16:05:05.481673 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349c518e-1293-47f1-b38e-b7c01b071146 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:05.499870 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349c518e-1293-47f1-b38e-b7c01b071146 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:05.601921 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b90774-666c-490a-b1f6-2c393cea3500 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:05.602662 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b90774-666c-490a-b1f6-2c393cea3500 admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:05.602943 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b90774-666c-490a-b1f6-2c393cea3500 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:05.603252 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b90774-666c-490a-b1f6-2c393cea3500 admin admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:05.603898 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b90774-666c-490a-b1f6-2c393cea3500 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:05.605544 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b90774-666c-490a-b1f6-2c393cea3500 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:05.619444 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1292/2557] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 30 16:05:05 2022] GET /identity/v3/roles => generated 1420 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:05:05.623354 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4712aa-5d41-4744-a2cc-1d7db825887c admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:05.636424 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4712aa-5d41-4744-a2cc-1d7db825887c admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:05.646261 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aac2570-678f-4e75-b029-956182b2b1ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:05.646942 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aac2570-678f-4e75-b029-956182b2b1ae admin admin] REQUEST_METHOD: `PUT` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:05.647218 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aac2570-678f-4e75-b029-956182b2b1ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:05.647484 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aac2570-678f-4e75-b029-956182b2b1ae admin admin] PATH_INFO: `/v3/projects/d6bbd8936e9645209e7af0e9dbe4e12a/users/c531981406634c20b211c3081034c036/roles/f265386950a641ac8413b53eda4ae34d` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:05.649462 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aac2570-678f-4e75-b029-956182b2b1ae admin admin] RBAC: Authorizing `identity:create_grant(project_id=d6bbd8936e9645209e7af0e9dbe4e12a, user_id=c531981406634c20b211c3081034c036, role_id=f265386950a641ac8413b53eda4ae34d)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:05.652116 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aac2570-678f-4e75-b029-956182b2b1ae admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:05.676801 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1266/2558] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 30 16:05:05 2022] PUT /identity/v3/projects/d6bbd8936e9645209e7af0e9dbe4e12a/users/c531981406634c20b211c3081034c036/roles/f265386950a641ac8413b53eda4ae34d => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:05:05.678715 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c256455-da70-49ab-9913-7eca5f5a976b None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:05.679172 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c256455-da70-49ab-9913-7eca5f5a976b None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:05.679514 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c256455-da70-49ab-9913-7eca5f5a976b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:05.711502 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-3c256455-da70-49ab-9913-7eca5f5a976b None None] MFA Rules not processed for user `c531981406634c20b211c3081034c036`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:05:05.712725 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-3c256455-da70-49ab-9913-7eca5f5a976b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:05:05.746328 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:05.797275 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1293/2559] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 16:05:05 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:05:05.803705 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aac2570-678f-4e75-b029-956182b2b1ae admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:05.806672 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-7aac2570-678f-4e75-b029-956182b2b1ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:05:05.877820 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aac2570-678f-4e75-b029-956182b2b1ae admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:05.882703 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d1688a-ea0c-4817-ae62-623a9f910909 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0100a04821804993866ea20f159663f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:05.883443 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d1688a-ea0c-4817-ae62-623a9f910909 service cinder] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:05.883716 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d1688a-ea0c-4817-ae62-623a9f910909 service cinder] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:05.883979 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d1688a-ea0c-4817-ae62-623a9f910909 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:05.959784 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d1688a-ea0c-4817-ae62-623a9f910909 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:05.961705 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d1688a-ea0c-4817-ae62-623a9f910909 service cinder] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:05.982076 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1267/2560] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 16:05:05 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:05:07.469823 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c256455-da70-49ab-9913-7eca5f5a976b None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:07.471883 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-3c256455-da70-49ab-9913-7eca5f5a976b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:05:07.534134 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c256455-da70-49ab-9913-7eca5f5a976b None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:07.540109 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a032802-db80-4433-921c-e1362862ca39 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f4157105f3488f807b7475008ab722', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:07.540848 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a032802-db80-4433-921c-e1362862ca39 service neutron] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:07.541111 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a032802-db80-4433-921c-e1362862ca39 service neutron] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:07.541428 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a032802-db80-4433-921c-e1362862ca39 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:07.543306 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-3a032802-db80-4433-921c-e1362862ca39 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:05:07.553517 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a032802-db80-4433-921c-e1362862ca39 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:07.554986 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a032802-db80-4433-921c-e1362862ca39 service neutron] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:07.563171 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1294/2561] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 16:05:07 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:05:07.689661 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d1688a-ea0c-4817-ae62-623a9f910909 service cinder] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:07.691114 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-69d1688a-ea0c-4817-ae62-623a9f910909 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:05:07.703059 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d1688a-ea0c-4817-ae62-623a9f910909 service cinder] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:07.706745 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d764c5da-18ed-4b49-b868-33443f42f880 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0100a04821804993866ea20f159663f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:07.707348 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d764c5da-18ed-4b49-b868-33443f42f880 service cinder] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:07.707619 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d764c5da-18ed-4b49-b868-33443f42f880 service cinder] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:07.707881 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d764c5da-18ed-4b49-b868-33443f42f880 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:07.743179 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:07.772113 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d764c5da-18ed-4b49-b868-33443f42f880 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:07.773684 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d764c5da-18ed-4b49-b868-33443f42f880 service cinder] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:07.783474 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1268/2562] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 16:05:07 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:05:08.370119 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a032802-db80-4433-921c-e1362862ca39 service neutron] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:08.380068 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a032802-db80-4433-921c-e1362862ca39 service neutron] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:08.383600 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a931c16-ae14-4f74-9e69-1f459b5251da service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0100a04821804993866ea20f159663f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:08.384217 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a931c16-ae14-4f74-9e69-1f459b5251da service cinder] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:08.384490 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a931c16-ae14-4f74-9e69-1f459b5251da service cinder] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:08.384754 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a931c16-ae14-4f74-9e69-1f459b5251da service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:08.394438 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a931c16-ae14-4f74-9e69-1f459b5251da service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:08.396996 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a931c16-ae14-4f74-9e69-1f459b5251da service cinder] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:08.407015 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1295/2563] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 16:05:08 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:05:12.414950 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d764c5da-18ed-4b49-b868-33443f42f880 service cinder] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:12.484594 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d764c5da-18ed-4b49-b868-33443f42f880 service cinder] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:12.490084 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa828ac-948d-496a-8385-915671b949c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:12.491261 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa828ac-948d-496a-8385-915671b949c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:12.491602 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa828ac-948d-496a-8385-915671b949c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:12.491912 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa828ac-948d-496a-8385-915671b949c0 admin admin] PATH_INFO: `/v3/users/e159d449db00454a802e5c4437a37f80` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:12.493718 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa828ac-948d-496a-8385-915671b949c0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e159d449db00454a802e5c4437a37f80)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:12.496106 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa828ac-948d-496a-8385-915671b949c0 admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:12.500927 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ad820a-f7e6-4453-bf2e-e874ae85e7de None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:12.501839 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ad820a-f7e6-4453-bf2e-e874ae85e7de None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:12.502274 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ad820a-f7e6-4453-bf2e-e874ae85e7de None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:12.503656 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1296/2564] 10.222.0.20 () {54 vars in 921 bytes} [Tue Aug 30 16:05:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 16:05:12.514817 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d084f86a-9371-4ae0-b388-a3b432df2f8e None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:12.515094 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d084f86a-9371-4ae0-b388-a3b432df2f8e None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:12.515360 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d084f86a-9371-4ae0-b388-a3b432df2f8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:12.538081 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-d084f86a-9371-4ae0-b388-a3b432df2f8e None None] MFA Rules not processed for user `7a1e756ae9be4a7bac1e81acf07e8f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:05:12.539421 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-d084f86a-9371-4ae0-b388-a3b432df2f8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:05:12.549085 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-ffa828ac-948d-496a-8385-915671b949c0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e159d449db00454a802e5c4437a37f80'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:05:12.559691 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-ffa828ac-948d-496a-8385-915671b949c0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e159d449db00454a802e5c4437a37f80'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:05:12.574745 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-ffa828ac-948d-496a-8385-915671b949c0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e159d449db00454a802e5c4437a37f80'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:05:12.586136 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-ffa828ac-948d-496a-8385-915671b949c0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e159d449db00454a802e5c4437a37f80'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:05:12.586957 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-ffa828ac-948d-496a-8385-915671b949c0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e159d449db00454a802e5c4437a37f80'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:05:12.594618 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1269/2565] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Aug 30 16:05:12 2022] DELETE /identity/v3/users/e159d449db00454a802e5c4437a37f80 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:05:12.600063 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c1256e-2062-4f78-b1cf-d33f2a821870 None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:12.600939 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c1256e-2062-4f78-b1cf-d33f2a821870 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:12.601317 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c1256e-2062-4f78-b1cf-d33f2a821870 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:12.619634 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1297/2566] 10.222.0.20 () {58 vars in 1018 bytes} [Tue Aug 30 16:05:12 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:05:12.623819 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-08c1256e-2062-4f78-b1cf-d33f2a821870 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:05:12.625060 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-08c1256e-2062-4f78-b1cf-d33f2a821870 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:05:12.635324 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d084f86a-9371-4ae0-b388-a3b432df2f8e None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:12.636825 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-d084f86a-9371-4ae0-b388-a3b432df2f8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:05:12.681636 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:12.688688 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:12.694600 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d084f86a-9371-4ae0-b388-a3b432df2f8e None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:12.697377 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1270/2567] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 16:05:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:05:12.704608 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c1256e-2062-4f78-b1cf-d33f2a821870 None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:12.706601 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-08c1256e-2062-4f78-b1cf-d33f2a821870 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:05:12.745709 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ba28dd-5855-4ed2-b641-1b204cf839c1 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b70e06a7d06d4199ab4fc36342ee1d68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:12.746244 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ba28dd-5855-4ed2-b641-1b204cf839c1 service glance] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:12.746519 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ba28dd-5855-4ed2-b641-1b204cf839c1 service glance] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:12.746785 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ba28dd-5855-4ed2-b641-1b204cf839c1 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:12.763957 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c1256e-2062-4f78-b1cf-d33f2a821870 None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:12.767671 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710af5b2-d5a7-40d1-85dd-8025208ad983 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f4157105f3488f807b7475008ab722', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:12.768279 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710af5b2-d5a7-40d1-85dd-8025208ad983 service neutron] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:12.768555 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710af5b2-d5a7-40d1-85dd-8025208ad983 service neutron] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:12.768824 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710af5b2-d5a7-40d1-85dd-8025208ad983 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:12.778142 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710af5b2-d5a7-40d1-85dd-8025208ad983 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:12.779649 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710af5b2-d5a7-40d1-85dd-8025208ad983 service neutron] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:12.789792 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1271/2568] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 16:05:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:05:12.815568 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ba28dd-5855-4ed2-b641-1b204cf839c1 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:12.817031 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ba28dd-5855-4ed2-b641-1b204cf839c1 service glance] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:12.830976 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1298/2569] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 16:05:12 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 16:05:12.912586 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1deb0a-91e4-4975-b43f-fac65dedcb47 None None] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:12.912875 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1deb0a-91e4-4975-b43f-fac65dedcb47 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:12.913148 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1deb0a-91e4-4975-b43f-fac65dedcb47 None None] PATH_INFO: `/` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:12.914553 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1272/2570] 10.222.0.20 () {54 vars in 921 bytes} [Tue Aug 30 16:05:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 16:05:12.919763 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524113f2-60d6-425c-855f-9b62e181512b None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:12.920102 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524113f2-60d6-425c-855f-9b62e181512b None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:12.920403 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524113f2-60d6-425c-855f-9b62e181512b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:12.943942 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-524113f2-60d6-425c-855f-9b62e181512b None None] MFA Rules not processed for user `65f4157105f3488f807b7475008ab722`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:05:12.945188 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-524113f2-60d6-425c-855f-9b62e181512b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:05:12.953606 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1299/2571] 10.222.0.20 () {58 vars in 1018 bytes} [Tue Aug 30 16:05:12 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 16:05:12.966017 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1deb0a-91e4-4975-b43f-fac65dedcb47 None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:12.978305 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1deb0a-91e4-4975-b43f-fac65dedcb47 None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:12.983017 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d7434a-abfa-478f-ae79-29375f78b9fa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f4157105f3488f807b7475008ab722', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:12.983628 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d7434a-abfa-478f-ae79-29375f78b9fa service neutron] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:12.983913 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d7434a-abfa-478f-ae79-29375f78b9fa service neutron] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:12.984190 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d7434a-abfa-478f-ae79-29375f78b9fa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:12.991589 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d7434a-abfa-478f-ae79-29375f78b9fa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:12.993207 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d7434a-abfa-478f-ae79-29375f78b9fa service neutron] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:13.001749 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1273/2572] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 16:05:12 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:05:13.067519 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524113f2-60d6-425c-855f-9b62e181512b None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:13.068989 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-524113f2-60d6-425c-855f-9b62e181512b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:05:13.084107 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524113f2-60d6-425c-855f-9b62e181512b None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:13.088941 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b7ce09-2589-4cfc-821f-ad20495205e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:13.089742 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b7ce09-2589-4cfc-821f-ad20495205e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:13.090121 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b7ce09-2589-4cfc-821f-ad20495205e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:13.090493 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b7ce09-2589-4cfc-821f-ad20495205e4 admin admin] PATH_INFO: `/v3/projects/a5457c7c1d774af4bb9a931a72091858` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:13.091753 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b7ce09-2589-4cfc-821f-ad20495205e4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a5457c7c1d774af4bb9a931a72091858)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:13.093903 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b7ce09-2589-4cfc-821f-ad20495205e4 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:13.094702 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf51a734-0ee1-4f3e-908a-e169dc56c413 None None] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:13.095580 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf51a734-0ee1-4f3e-908a-e169dc56c413 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:13.096217 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf51a734-0ee1-4f3e-908a-e169dc56c413 None None] PATH_INFO: `/` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:13.098496 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1274/2573] 10.222.0.20 () {54 vars in 921 bytes} [Tue Aug 30 16:05:13 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 16:05:13.108467 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229619f0-bce4-4cc1-b171-8231ecc19119 None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:13.108818 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229619f0-bce4-4cc1-b171-8231ecc19119 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:13.109099 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229619f0-bce4-4cc1-b171-8231ecc19119 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:13.131875 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-229619f0-bce4-4cc1-b171-8231ecc19119 None None] MFA Rules not processed for user `7a1e756ae9be4a7bac1e81acf07e8f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:05:13.133118 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-229619f0-bce4-4cc1-b171-8231ecc19119 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:05:13.145528 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-12b7ce09-2589-4cfc-821f-ad20495205e4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a5457c7c1d774af4bb9a931a72091858', 'request_id': 'req-12b7ce09-2589-4cfc-821f-ad20495205e4'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:05:13.155319 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-12b7ce09-2589-4cfc-821f-ad20495205e4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a5457c7c1d774af4bb9a931a72091858', 'request_id': 'req-12b7ce09-2589-4cfc-821f-ad20495205e4'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:05:13.162139 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-12b7ce09-2589-4cfc-821f-ad20495205e4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a5457c7c1d774af4bb9a931a72091858', 'request_id': 'req-12b7ce09-2589-4cfc-821f-ad20495205e4'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:05:13.172749 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-12b7ce09-2589-4cfc-821f-ad20495205e4 admin admin] The token cache is being invalidate because project a5457c7c1d774af4bb9a931a72091858 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81511) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 16:05:13.173036 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-12b7ce09-2589-4cfc-821f-ad20495205e4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:05:13.173736 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-12b7ce09-2589-4cfc-821f-ad20495205e4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:05:13.175432 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1300/2574] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Aug 30 16:05:13 2022] DELETE /identity/v3/projects/a5457c7c1d774af4bb9a931a72091858 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:05:13.179985 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b7ce09-2589-4cfc-821f-ad20495205e4 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:13.183216 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-12b7ce09-2589-4cfc-821f-ad20495205e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:05:13.206216 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1275/2575] 10.222.0.20 () {58 vars in 1018 bytes} [Tue Aug 30 16:05:13 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 16:05:13.210209 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:13.217922 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229619f0-bce4-4cc1-b171-8231ecc19119 None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:13.219535 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-229619f0-bce4-4cc1-b171-8231ecc19119 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:05:13.260648 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b7ce09-2589-4cfc-821f-ad20495205e4 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:13.267541 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8820f0-c0d4-4d27-95e2-8763ba031ccc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:13.268541 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8820f0-c0d4-4d27-95e2-8763ba031ccc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:13.268541 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8820f0-c0d4-4d27-95e2-8763ba031ccc admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:13.268949 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8820f0-c0d4-4d27-95e2-8763ba031ccc admin admin] PATH_INFO: `/v3/users/1950c274d93b480bad1e8ef0d2438b4e` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:13.270208 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8820f0-c0d4-4d27-95e2-8763ba031ccc admin admin] RBAC: Authorizing `identity:delete_user(user_id=1950c274d93b480bad1e8ef0d2438b4e)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:13.272352 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8820f0-c0d4-4d27-95e2-8763ba031ccc admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:13.295682 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229619f0-bce4-4cc1-b171-8231ecc19119 None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:13.299879 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1914d80-6877-4881-9043-0c08342ebdb1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f4157105f3488f807b7475008ab722', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:13.300505 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1914d80-6877-4881-9043-0c08342ebdb1 service neutron] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:13.300933 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1914d80-6877-4881-9043-0c08342ebdb1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:13.301211 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1914d80-6877-4881-9043-0c08342ebdb1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:13.313579 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1914d80-6877-4881-9043-0c08342ebdb1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:13.315250 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1914d80-6877-4881-9043-0c08342ebdb1 service neutron] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:13.330439 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-aa8820f0-c0d4-4d27-95e2-8763ba031ccc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1950c274d93b480bad1e8ef0d2438b4e'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:05:13.331062 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1276/2576] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 16:05:13 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:05:13.364768 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-aa8820f0-c0d4-4d27-95e2-8763ba031ccc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1950c274d93b480bad1e8ef0d2438b4e'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:05:13.374985 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-aa8820f0-c0d4-4d27-95e2-8763ba031ccc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1950c274d93b480bad1e8ef0d2438b4e'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:05:13.375607 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-aa8820f0-c0d4-4d27-95e2-8763ba031ccc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1950c274d93b480bad1e8ef0d2438b4e'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:05:13.396996 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-aa8820f0-c0d4-4d27-95e2-8763ba031ccc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1950c274d93b480bad1e8ef0d2438b4e'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:05:13.404282 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1301/2577] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Aug 30 16:05:13 2022] DELETE /identity/v3/users/1950c274d93b480bad1e8ef0d2438b4e => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:05:13.416222 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1914d80-6877-4881-9043-0c08342ebdb1 service neutron] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:13.417682 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-b1914d80-6877-4881-9043-0c08342ebdb1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:05:13.442379 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:13.490830 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1914d80-6877-4881-9043-0c08342ebdb1 service neutron] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:13.552525 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6caac6d4-8c4c-40f1-bb6f-dd7320cf255b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc12c6d31eab4a17a160250adc2380c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:13.553498 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6caac6d4-8c4c-40f1-bb6f-dd7320cf255b service placement] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:13.553498 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6caac6d4-8c4c-40f1-bb6f-dd7320cf255b service placement] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:13.553712 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6caac6d4-8c4c-40f1-bb6f-dd7320cf255b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:13.555592 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-6caac6d4-8c4c-40f1-bb6f-dd7320cf255b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:05:13.570293 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6caac6d4-8c4c-40f1-bb6f-dd7320cf255b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:13.571957 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6caac6d4-8c4c-40f1-bb6f-dd7320cf255b service placement] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:13.582132 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1277/2578] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 16:05:13 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:05:13.695496 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8820f0-c0d4-4d27-95e2-8763ba031ccc admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:13.696847 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-aa8820f0-c0d4-4d27-95e2-8763ba031ccc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:05:13.771078 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8820f0-c0d4-4d27-95e2-8763ba031ccc admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:13.777239 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602133ac-41fe-46e2-92c0-477f64ec4b54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:13.777903 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602133ac-41fe-46e2-92c0-477f64ec4b54 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:13.778274 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602133ac-41fe-46e2-92c0-477f64ec4b54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:13.778487 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602133ac-41fe-46e2-92c0-477f64ec4b54 admin admin] PATH_INFO: `/v3/projects/d8e35fde3e29488e82dd5ac977cc2bd8` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:13.779606 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602133ac-41fe-46e2-92c0-477f64ec4b54 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d8e35fde3e29488e82dd5ac977cc2bd8)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:13.781689 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602133ac-41fe-46e2-92c0-477f64ec4b54 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:13.830206 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:13.839118 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-602133ac-41fe-46e2-92c0-477f64ec4b54 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd8e35fde3e29488e82dd5ac977cc2bd8', 'request_id': 'req-602133ac-41fe-46e2-92c0-477f64ec4b54'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:05:13.852499 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-602133ac-41fe-46e2-92c0-477f64ec4b54 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd8e35fde3e29488e82dd5ac977cc2bd8', 'request_id': 'req-602133ac-41fe-46e2-92c0-477f64ec4b54'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:05:13.860905 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-602133ac-41fe-46e2-92c0-477f64ec4b54 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd8e35fde3e29488e82dd5ac977cc2bd8', 'request_id': 'req-602133ac-41fe-46e2-92c0-477f64ec4b54'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:05:13.871572 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-602133ac-41fe-46e2-92c0-477f64ec4b54 admin admin] The token cache is being invalidate because project d8e35fde3e29488e82dd5ac977cc2bd8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81511) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 16:05:13.871873 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-602133ac-41fe-46e2-92c0-477f64ec4b54 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:05:13.872536 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-602133ac-41fe-46e2-92c0-477f64ec4b54 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:05:13.874097 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1302/2579] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Aug 30 16:05:13 2022] DELETE /identity/v3/projects/d8e35fde3e29488e82dd5ac977cc2bd8 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:05:13.878147 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6caac6d4-8c4c-40f1-bb6f-dd7320cf255b service placement] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:13.888411 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6caac6d4-8c4c-40f1-bb6f-dd7320cf255b service placement] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:13.945862 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533b8b6d-d31b-4974-b549-fab83cae9e1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:13.946497 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533b8b6d-d31b-4974-b549-fab83cae9e1b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:13.946780 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533b8b6d-d31b-4974-b549-fab83cae9e1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:13.947083 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533b8b6d-d31b-4974-b549-fab83cae9e1b admin admin] PATH_INFO: `/v3/users/c531981406634c20b211c3081034c036` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:13.948177 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533b8b6d-d31b-4974-b549-fab83cae9e1b admin admin] RBAC: Authorizing `identity:delete_user(user_id=c531981406634c20b211c3081034c036)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:13.950167 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533b8b6d-d31b-4974-b549-fab83cae9e1b admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:14.002544 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-533b8b6d-d31b-4974-b549-fab83cae9e1b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c531981406634c20b211c3081034c036'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:05:14.013347 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-533b8b6d-d31b-4974-b549-fab83cae9e1b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c531981406634c20b211c3081034c036'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:05:14.014690 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:14.029364 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-533b8b6d-d31b-4974-b549-fab83cae9e1b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c531981406634c20b211c3081034c036'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:05:14.036532 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-533b8b6d-d31b-4974-b549-fab83cae9e1b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c531981406634c20b211c3081034c036'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:05:14.037392 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-533b8b6d-d31b-4974-b549-fab83cae9e1b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c531981406634c20b211c3081034c036'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:05:14.043813 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1278/2580] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Aug 30 16:05:13 2022] DELETE /identity/v3/users/c531981406634c20b211c3081034c036 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:05:14.319543 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602133ac-41fe-46e2-92c0-477f64ec4b54 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:14.321014 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-602133ac-41fe-46e2-92c0-477f64ec4b54 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:05:14.336358 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602133ac-41fe-46e2-92c0-477f64ec4b54 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:14.398951 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144516b6-84a6-47d7-97cf-3d569f67b2a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:14.399623 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144516b6-84a6-47d7-97cf-3d569f67b2a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:14.399902 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144516b6-84a6-47d7-97cf-3d569f67b2a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:14.400160 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144516b6-84a6-47d7-97cf-3d569f67b2a5 admin admin] PATH_INFO: `/v3/projects/d6bbd8936e9645209e7af0e9dbe4e12a` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:14.401303 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144516b6-84a6-47d7-97cf-3d569f67b2a5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d6bbd8936e9645209e7af0e9dbe4e12a)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:14.413040 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144516b6-84a6-47d7-97cf-3d569f67b2a5 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:14.464383 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-144516b6-84a6-47d7-97cf-3d569f67b2a5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd6bbd8936e9645209e7af0e9dbe4e12a', 'request_id': 'req-144516b6-84a6-47d7-97cf-3d569f67b2a5'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:05:14.473547 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-144516b6-84a6-47d7-97cf-3d569f67b2a5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd6bbd8936e9645209e7af0e9dbe4e12a', 'request_id': 'req-144516b6-84a6-47d7-97cf-3d569f67b2a5'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:05:14.480514 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-144516b6-84a6-47d7-97cf-3d569f67b2a5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd6bbd8936e9645209e7af0e9dbe4e12a', 'request_id': 'req-144516b6-84a6-47d7-97cf-3d569f67b2a5'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:05:14.491970 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-144516b6-84a6-47d7-97cf-3d569f67b2a5 admin admin] The token cache is being invalidate because project d6bbd8936e9645209e7af0e9dbe4e12a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81511) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 16:05:14.492257 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-144516b6-84a6-47d7-97cf-3d569f67b2a5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:05:14.492896 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-144516b6-84a6-47d7-97cf-3d569f67b2a5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:05:14.494416 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1303/2581] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Aug 30 16:05:14 2022] DELETE /identity/v3/projects/d6bbd8936e9645209e7af0e9dbe4e12a => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:05:14.499430 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a386aaf-f3f1-4cbc-901e-b9986c80a81f None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:14.499916 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a386aaf-f3f1-4cbc-901e-b9986c80a81f None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:14.500110 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a386aaf-f3f1-4cbc-901e-b9986c80a81f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:14.520062 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-5a386aaf-f3f1-4cbc-901e-b9986c80a81f None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:05:14.521288 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-5a386aaf-f3f1-4cbc-901e-b9986c80a81f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:05:14.584203 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1279/2582] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 16:05:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 16:05:14.592280 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f9c051-256b-4dc5-bbc4-1e6477356028 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:14.592574 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f9c051-256b-4dc5-bbc4-1e6477356028 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:14.592918 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f9c051-256b-4dc5-bbc4-1e6477356028 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:14.614033 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-72f9c051-256b-4dc5-bbc4-1e6477356028 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:05:14.615289 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-72f9c051-256b-4dc5-bbc4-1e6477356028 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:05:14.630746 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1304/2583] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 16:05:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:05:14.631166 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a386aaf-f3f1-4cbc-901e-b9986c80a81f None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:14.641810 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a386aaf-f3f1-4cbc-901e-b9986c80a81f None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:14.648021 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8c1d93-40ec-4b58-acd1-8b1dc9b703ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:14.648650 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8c1d93-40ec-4b58-acd1-8b1dc9b703ab admin admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:14.648933 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8c1d93-40ec-4b58-acd1-8b1dc9b703ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:14.649205 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8c1d93-40ec-4b58-acd1-8b1dc9b703ab admin admin] PATH_INFO: `/v3/domains` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:14.649909 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8c1d93-40ec-4b58-acd1-8b1dc9b703ab admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81512) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:05:14.650212 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8c1d93-40ec-4b58-acd1-8b1dc9b703ab admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:14.651874 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8c1d93-40ec-4b58-acd1-8b1dc9b703ab admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:14.664421 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1280/2584] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Aug 30 16:05:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 16:05:14.668918 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f9c051-256b-4dc5-bbc4-1e6477356028 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:14.678474 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f9c051-256b-4dc5-bbc4-1e6477356028 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:14.683233 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226842e0-5fa2-4e40-b19d-234118da39b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:14.683838 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226842e0-5fa2-4e40-b19d-234118da39b6 admin admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:14.684117 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226842e0-5fa2-4e40-b19d-234118da39b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:14.684388 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226842e0-5fa2-4e40-b19d-234118da39b6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:14.685030 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226842e0-5fa2-4e40-b19d-234118da39b6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:14.687036 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226842e0-5fa2-4e40-b19d-234118da39b6 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:14.718272 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1305/2585] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Aug 30 16:05:14 2022] POST /identity/v3/projects => generated 371 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:05:14.722741 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8c1d93-40ec-4b58-acd1-8b1dc9b703ab admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:14.733785 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8c1d93-40ec-4b58-acd1-8b1dc9b703ab admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:14.791789 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37543e2f-5583-4263-a58e-cce97622784f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:14.792405 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37543e2f-5583-4263-a58e-cce97622784f admin admin] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:14.792683 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37543e2f-5583-4263-a58e-cce97622784f admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:14.792954 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37543e2f-5583-4263-a58e-cce97622784f admin admin] PATH_INFO: `/v3/users` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:14.793626 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37543e2f-5583-4263-a58e-cce97622784f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:14.795606 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37543e2f-5583-4263-a58e-cce97622784f admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:14.818019 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1281/2586] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Aug 30 16:05:14 2022] POST /identity/v3/users => generated 286 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:05:14.822105 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226842e0-5fa2-4e40-b19d-234118da39b6 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:14.832794 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226842e0-5fa2-4e40-b19d-234118da39b6 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:14.838612 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ceec00-704f-4d5d-978b-a74f4ddda27c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:14.839934 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ceec00-704f-4d5d-978b-a74f4ddda27c admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:14.840225 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ceec00-704f-4d5d-978b-a74f4ddda27c admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:14.840482 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ceec00-704f-4d5d-978b-a74f4ddda27c admin admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:14.841128 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ceec00-704f-4d5d-978b-a74f4ddda27c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:14.842777 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ceec00-704f-4d5d-978b-a74f4ddda27c admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:14.853071 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1306/2587] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 30 16:05:14 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:05:14.857742 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37543e2f-5583-4263-a58e-cce97622784f admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:14.870170 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37543e2f-5583-4263-a58e-cce97622784f admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:14.882641 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81ff82c-c94f-4744-b482-8c9bab97a0b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:14.883265 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81ff82c-c94f-4744-b482-8c9bab97a0b7 admin admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:14.883539 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81ff82c-c94f-4744-b482-8c9bab97a0b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:14.883796 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81ff82c-c94f-4744-b482-8c9bab97a0b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:14.884533 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81ff82c-c94f-4744-b482-8c9bab97a0b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:14.886164 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81ff82c-c94f-4744-b482-8c9bab97a0b7 admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:14.897749 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1282/2588] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 30 16:05:14 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 16:05:14.901960 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ceec00-704f-4d5d-978b-a74f4ddda27c admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:14.912587 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ceec00-704f-4d5d-978b-a74f4ddda27c admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:14.917514 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e94ebd-7e15-4b19-9fa1-0bf58b3872ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:14.918294 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e94ebd-7e15-4b19-9fa1-0bf58b3872ac admin admin] REQUEST_METHOD: `PUT` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:14.918691 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e94ebd-7e15-4b19-9fa1-0bf58b3872ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:14.919028 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e94ebd-7e15-4b19-9fa1-0bf58b3872ac admin admin] PATH_INFO: `/v3/projects/ce96dc27712e4c12b7b1566a6c78ac5f/users/bd914c4798094e34b1e588ade55fc06c/roles/df8f9f844b1b439181d75a07c1a93418` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:14.935226 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e94ebd-7e15-4b19-9fa1-0bf58b3872ac admin admin] RBAC: Authorizing `identity:create_grant(project_id=ce96dc27712e4c12b7b1566a6c78ac5f, user_id=bd914c4798094e34b1e588ade55fc06c, role_id=df8f9f844b1b439181d75a07c1a93418)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:14.938162 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e94ebd-7e15-4b19-9fa1-0bf58b3872ac admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:14.941493 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=85,keystone:DELETE=11,keystone:INSERT=4 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:14.953032 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1307/2589] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 30 16:05:14 2022] PUT /identity/v3/projects/ce96dc27712e4c12b7b1566a6c78ac5f/users/bd914c4798094e34b1e588ade55fc06c/roles/df8f9f844b1b439181d75a07c1a93418 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 16:05:14.965037 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e851d5c-64af-4a90-b627-766fe9cafc59 None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:14.965328 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e851d5c-64af-4a90-b627-766fe9cafc59 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:14.965589 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e851d5c-64af-4a90-b627-766fe9cafc59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:15.002324 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-9e851d5c-64af-4a90-b627-766fe9cafc59 None None] MFA Rules not processed for user `bd914c4798094e34b1e588ade55fc06c`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:05:15.003963 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-9e851d5c-64af-4a90-b627-766fe9cafc59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:05:15.036119 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:15.083950 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1283/2590] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 16:05:14 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:05:15.096130 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e94ebd-7e15-4b19-9fa1-0bf58b3872ac admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:15.098025 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-14e94ebd-7e15-4b19-9fa1-0bf58b3872ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:05:15.160408 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e94ebd-7e15-4b19-9fa1-0bf58b3872ac admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:15.164112 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455a9b41-f1d5-45f2-a7d2-a4ed5e514847 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b70e06a7d06d4199ab4fc36342ee1d68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:15.164720 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455a9b41-f1d5-45f2-a7d2-a4ed5e514847 service glance] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:15.164993 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455a9b41-f1d5-45f2-a7d2-a4ed5e514847 service glance] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:15.165335 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455a9b41-f1d5-45f2-a7d2-a4ed5e514847 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:15.173695 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455a9b41-f1d5-45f2-a7d2-a4ed5e514847 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:15.175125 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455a9b41-f1d5-45f2-a7d2-a4ed5e514847 service glance] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:15.184250 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1308/2591] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 16:05:15 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 16:05:15.540067 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85b984c-9f50-4167-bf08-1e73257947ad None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:15.540634 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85b984c-9f50-4167-bf08-1e73257947ad None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:15.540814 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85b984c-9f50-4167-bf08-1e73257947ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:15.562322 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-b85b984c-9f50-4167-bf08-1e73257947ad None None] MFA Rules not processed for user `07130d598131443486c2bc1c6673c45b`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:05:15.563541 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-b85b984c-9f50-4167-bf08-1e73257947ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:05:15.656741 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1284/2592] 10.222.0.20 () {58 vars in 1009 bytes} [Tue Aug 30 16:05:15 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:05:15.670306 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455a9b41-f1d5-45f2-a7d2-a4ed5e514847 service glance] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:15.671754 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-455a9b41-f1d5-45f2-a7d2-a4ed5e514847 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:05:15.751584 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455a9b41-f1d5-45f2-a7d2-a4ed5e514847 service glance] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:15.756953 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0957a7-f863-4ed2-94e1-46dac3e6737d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8fdafdc39a594bb4b0aeaf2fbc84c2fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:15.757868 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0957a7-f863-4ed2-94e1-46dac3e6737d service swift] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:15.758245 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0957a7-f863-4ed2-94e1-46dac3e6737d service swift] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:15.758582 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0957a7-f863-4ed2-94e1-46dac3e6737d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:15.767636 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0957a7-f863-4ed2-94e1-46dac3e6737d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:15.769350 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0957a7-f863-4ed2-94e1-46dac3e6737d service swift] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:15.777356 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1309/2593] 10.222.0.20 () {60 vars in 1384 bytes} [Tue Aug 30 16:05:15 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 30 16:05:25.646871 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=40 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:25.774164 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=60 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:33.196234 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de44aef3-ac7c-4282-892c-e4c50aad7813 None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:33.197343 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de44aef3-ac7c-4282-892c-e4c50aad7813 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:33.197774 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de44aef3-ac7c-4282-892c-e4c50aad7813 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:33.218546 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-de44aef3-ac7c-4282-892c-e4c50aad7813 None None] MFA Rules not processed for user `07130d598131443486c2bc1c6673c45b`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:05:33.219973 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-de44aef3-ac7c-4282-892c-e4c50aad7813 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:05:33.228106 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1285/2594] 10.222.0.20 () {58 vars in 967 bytes} [Tue Aug 30 16:05:33 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:05:33.241561 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0957a7-f863-4ed2-94e1-46dac3e6737d service swift] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:33.255269 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0957a7-f863-4ed2-94e1-46dac3e6737d service swift] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:33.262294 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4c7300-c1e6-4972-9e3d-1f1c17efcfe7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8fdafdc39a594bb4b0aeaf2fbc84c2fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:33.262940 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4c7300-c1e6-4972-9e3d-1f1c17efcfe7 service swift] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:33.263216 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4c7300-c1e6-4972-9e3d-1f1c17efcfe7 service swift] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:33.263621 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4c7300-c1e6-4972-9e3d-1f1c17efcfe7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:33.272890 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4c7300-c1e6-4972-9e3d-1f1c17efcfe7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:33.274594 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4c7300-c1e6-4972-9e3d-1f1c17efcfe7 service swift] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:33.283627 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1310/2595] 10.222.0.20 () {60 vars in 1384 bytes} [Tue Aug 30 16:05:33 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 30 16:05:33.386373 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de44aef3-ac7c-4282-892c-e4c50aad7813 None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:33.387237 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-de44aef3-ac7c-4282-892c-e4c50aad7813 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:05:33.506058 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de44aef3-ac7c-4282-892c-e4c50aad7813 None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:33.513979 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b912f38-bbfb-45fe-b3dc-55b9a5a243aa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a1e756ae9be4a7bac1e81acf07e8f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:33.515100 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b912f38-bbfb-45fe-b3dc-55b9a5a243aa service nova] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:33.516838 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b912f38-bbfb-45fe-b3dc-55b9a5a243aa service nova] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:33.517450 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b912f38-bbfb-45fe-b3dc-55b9a5a243aa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:33.522791 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-8b912f38-bbfb-45fe-b3dc-55b9a5a243aa service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:05:33.552265 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b912f38-bbfb-45fe-b3dc-55b9a5a243aa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:33.562995 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b912f38-bbfb-45fe-b3dc-55b9a5a243aa service nova] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:33.593942 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1286/2596] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 16:05:33 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:05:41.502116 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4c7300-c1e6-4972-9e3d-1f1c17efcfe7 service swift] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:41.503569 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-ff4c7300-c1e6-4972-9e3d-1f1c17efcfe7 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:05:41.567359 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4c7300-c1e6-4972-9e3d-1f1c17efcfe7 service swift] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:41.571370 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134833d1-974a-47cd-8a24-a97e813927b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f4157105f3488f807b7475008ab722', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:41.571982 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134833d1-974a-47cd-8a24-a97e813927b0 service neutron] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:41.572263 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134833d1-974a-47cd-8a24-a97e813927b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:41.572529 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134833d1-974a-47cd-8a24-a97e813927b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:41.574360 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-134833d1-974a-47cd-8a24-a97e813927b0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:05:41.645835 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134833d1-974a-47cd-8a24-a97e813927b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:41.647869 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134833d1-974a-47cd-8a24-a97e813927b0 service neutron] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:41.657303 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b912f38-bbfb-45fe-b3dc-55b9a5a243aa service nova] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:41.661455 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1311/2597] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 16:05:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:05:41.668731 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b912f38-bbfb-45fe-b3dc-55b9a5a243aa service nova] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:41.674444 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c4bd71-37a6-4a53-9512-d2004dfdc6f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:41.675081 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c4bd71-37a6-4a53-9512-d2004dfdc6f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:41.675390 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c4bd71-37a6-4a53-9512-d2004dfdc6f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:41.675852 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c4bd71-37a6-4a53-9512-d2004dfdc6f8 admin admin] PATH_INFO: `/v3/users/bd914c4798094e34b1e588ade55fc06c` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:41.677065 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c4bd71-37a6-4a53-9512-d2004dfdc6f8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bd914c4798094e34b1e588ade55fc06c)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:41.679255 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c4bd71-37a6-4a53-9512-d2004dfdc6f8 admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:41.734845 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-66c4bd71-37a6-4a53-9512-d2004dfdc6f8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bd914c4798094e34b1e588ade55fc06c'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:05:41.743991 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-66c4bd71-37a6-4a53-9512-d2004dfdc6f8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bd914c4798094e34b1e588ade55fc06c'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:05:41.756502 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-66c4bd71-37a6-4a53-9512-d2004dfdc6f8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bd914c4798094e34b1e588ade55fc06c'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:05:41.772131 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-66c4bd71-37a6-4a53-9512-d2004dfdc6f8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bd914c4798094e34b1e588ade55fc06c'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:05:41.772994 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-66c4bd71-37a6-4a53-9512-d2004dfdc6f8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bd914c4798094e34b1e588ade55fc06c'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:05:41.779344 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1287/2598] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Aug 30 16:05:41 2022] DELETE /identity/v3/users/bd914c4798094e34b1e588ade55fc06c => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:05:41.784175 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25aea8ab-af47-47b7-99e8-320f052ad5b5 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:41.784458 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25aea8ab-af47-47b7-99e8-320f052ad5b5 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:41.784790 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25aea8ab-af47-47b7-99e8-320f052ad5b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:41.804505 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-25aea8ab-af47-47b7-99e8-320f052ad5b5 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:05:41.805762 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-25aea8ab-af47-47b7-99e8-320f052ad5b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:05:41.865586 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1312/2599] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 16:05:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 16:05:41.875496 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c4bd71-37a6-4a53-9512-d2004dfdc6f8 admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:41.877047 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-66c4bd71-37a6-4a53-9512-d2004dfdc6f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:05:41.936188 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c4bd71-37a6-4a53-9512-d2004dfdc6f8 admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:41.940215 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61577aec-bc49-4d66-81a7-73e115985c90 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f4157105f3488f807b7475008ab722', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:41.940826 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61577aec-bc49-4d66-81a7-73e115985c90 service neutron] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:41.941111 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61577aec-bc49-4d66-81a7-73e115985c90 service neutron] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:41.941409 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61577aec-bc49-4d66-81a7-73e115985c90 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:41.948820 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:41.952032 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61577aec-bc49-4d66-81a7-73e115985c90 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:41.953845 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61577aec-bc49-4d66-81a7-73e115985c90 service neutron] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:41.965140 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1288/2600] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 16:05:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:05:42.258124 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25aea8ab-af47-47b7-99e8-320f052ad5b5 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:42.259798 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-25aea8ab-af47-47b7-99e8-320f052ad5b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:05:42.281934 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25aea8ab-af47-47b7-99e8-320f052ad5b5 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:42.289926 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5088a2-26b5-4afb-a92e-28e5b13bb1a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:42.290958 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5088a2-26b5-4afb-a92e-28e5b13bb1a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:42.291404 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5088a2-26b5-4afb-a92e-28e5b13bb1a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:42.291674 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5088a2-26b5-4afb-a92e-28e5b13bb1a1 admin admin] PATH_INFO: `/v3/projects/ce96dc27712e4c12b7b1566a6c78ac5f` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:42.292841 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5088a2-26b5-4afb-a92e-28e5b13bb1a1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ce96dc27712e4c12b7b1566a6c78ac5f)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:42.294807 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5088a2-26b5-4afb-a92e-28e5b13bb1a1 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:42.332661 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:42.345940 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-bb5088a2-26b5-4afb-a92e-28e5b13bb1a1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ce96dc27712e4c12b7b1566a6c78ac5f', 'request_id': 'req-bb5088a2-26b5-4afb-a92e-28e5b13bb1a1'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:05:42.354505 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-bb5088a2-26b5-4afb-a92e-28e5b13bb1a1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ce96dc27712e4c12b7b1566a6c78ac5f', 'request_id': 'req-bb5088a2-26b5-4afb-a92e-28e5b13bb1a1'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:05:42.361374 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-bb5088a2-26b5-4afb-a92e-28e5b13bb1a1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ce96dc27712e4c12b7b1566a6c78ac5f', 'request_id': 'req-bb5088a2-26b5-4afb-a92e-28e5b13bb1a1'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:05:42.371599 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-bb5088a2-26b5-4afb-a92e-28e5b13bb1a1 admin admin] The token cache is being invalidate because project ce96dc27712e4c12b7b1566a6c78ac5f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81511) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 16:05:42.371885 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-bb5088a2-26b5-4afb-a92e-28e5b13bb1a1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:05:42.372533 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-bb5088a2-26b5-4afb-a92e-28e5b13bb1a1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:05:42.374061 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1313/2601] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Aug 30 16:05:42 2022] DELETE /identity/v3/projects/ce96dc27712e4c12b7b1566a6c78ac5f => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:05:42.378897 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34133e5-401c-495b-a22a-35e91f0284f0 None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:42.379210 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34133e5-401c-495b-a22a-35e91f0284f0 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:42.379478 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34133e5-401c-495b-a22a-35e91f0284f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:42.399624 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-b34133e5-401c-495b-a22a-35e91f0284f0 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:05:42.400806 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-b34133e5-401c-495b-a22a-35e91f0284f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:05:42.461005 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1289/2602] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 16:05:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:05:42.469755 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c88e99a-ddf6-4210-8959-a415d1d2d680 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:42.470031 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c88e99a-ddf6-4210-8959-a415d1d2d680 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:42.470306 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c88e99a-ddf6-4210-8959-a415d1d2d680 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:42.492073 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-4c88e99a-ddf6-4210-8959-a415d1d2d680 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:05:42.493330 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-4c88e99a-ddf6-4210-8959-a415d1d2d680 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:05:42.501163 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1314/2603] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 16:05:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:05:42.505559 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34133e5-401c-495b-a22a-35e91f0284f0 None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:42.515719 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34133e5-401c-495b-a22a-35e91f0284f0 None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:42.520868 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa9910d-0f43-4e50-9324-427dd83e14dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:42.521493 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa9910d-0f43-4e50-9324-427dd83e14dc admin admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:42.521782 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa9910d-0f43-4e50-9324-427dd83e14dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:42.522052 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa9910d-0f43-4e50-9324-427dd83e14dc admin admin] PATH_INFO: `/v3/domains` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:42.522769 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa9910d-0f43-4e50-9324-427dd83e14dc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81512) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:05:42.523063 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa9910d-0f43-4e50-9324-427dd83e14dc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:42.524720 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa9910d-0f43-4e50-9324-427dd83e14dc admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:42.537244 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1290/2604] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Aug 30 16:05:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:05:42.542188 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c88e99a-ddf6-4210-8959-a415d1d2d680 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:42.551758 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c88e99a-ddf6-4210-8959-a415d1d2d680 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:42.560124 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65906726-bbf0-4283-9e5e-6dcf0fb95377 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:42.560766 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65906726-bbf0-4283-9e5e-6dcf0fb95377 admin admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:42.561040 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65906726-bbf0-4283-9e5e-6dcf0fb95377 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:42.561359 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65906726-bbf0-4283-9e5e-6dcf0fb95377 admin admin] PATH_INFO: `/v3/projects` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:42.562039 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65906726-bbf0-4283-9e5e-6dcf0fb95377 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:42.564392 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65906726-bbf0-4283-9e5e-6dcf0fb95377 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:42.598271 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1315/2605] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Aug 30 16:05:42 2022] POST /identity/v3/projects => generated 383 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:05:42.602413 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa9910d-0f43-4e50-9324-427dd83e14dc admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:42.613680 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa9910d-0f43-4e50-9324-427dd83e14dc admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:42.690578 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508df4b4-62cf-48db-a785-4664ffa994af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:42.691288 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508df4b4-62cf-48db-a785-4664ffa994af admin admin] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:42.691530 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508df4b4-62cf-48db-a785-4664ffa994af admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:42.691744 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508df4b4-62cf-48db-a785-4664ffa994af admin admin] PATH_INFO: `/v3/users` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:42.692382 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508df4b4-62cf-48db-a785-4664ffa994af admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:42.696156 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508df4b4-62cf-48db-a785-4664ffa994af admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:42.720666 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1291/2606] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Aug 30 16:05:42 2022] POST /identity/v3/users => generated 292 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:05:42.725481 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65906726-bbf0-4283-9e5e-6dcf0fb95377 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:42.737496 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65906726-bbf0-4283-9e5e-6dcf0fb95377 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:42.742509 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9063d61e-6439-4ac9-8f4b-ac38fec0e776 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:42.743233 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9063d61e-6439-4ac9-8f4b-ac38fec0e776 admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:42.743612 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9063d61e-6439-4ac9-8f4b-ac38fec0e776 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:42.743990 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9063d61e-6439-4ac9-8f4b-ac38fec0e776 admin admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:42.744715 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9063d61e-6439-4ac9-8f4b-ac38fec0e776 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:42.746446 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9063d61e-6439-4ac9-8f4b-ac38fec0e776 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:42.757951 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1316/2607] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 30 16:05:42 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 16:05:42.761919 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508df4b4-62cf-48db-a785-4664ffa994af admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:42.772088 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508df4b4-62cf-48db-a785-4664ffa994af admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:42.777071 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46015c5f-b5b9-4f9d-91c8-98cec48d2203 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:42.777809 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46015c5f-b5b9-4f9d-91c8-98cec48d2203 admin admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:42.778153 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46015c5f-b5b9-4f9d-91c8-98cec48d2203 admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:42.778489 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46015c5f-b5b9-4f9d-91c8-98cec48d2203 admin admin] PATH_INFO: `/v3/roles` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:42.779143 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46015c5f-b5b9-4f9d-91c8-98cec48d2203 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:42.780980 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46015c5f-b5b9-4f9d-91c8-98cec48d2203 admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:42.792765 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1292/2608] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 30 16:05:42 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 16:05:42.796947 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9063d61e-6439-4ac9-8f4b-ac38fec0e776 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:42.807347 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9063d61e-6439-4ac9-8f4b-ac38fec0e776 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:42.812434 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f894d1-3a54-4ebb-ab64-746e3548a40f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:42.813079 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f894d1-3a54-4ebb-ab64-746e3548a40f admin admin] REQUEST_METHOD: `PUT` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:42.813423 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f894d1-3a54-4ebb-ab64-746e3548a40f admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:42.813699 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f894d1-3a54-4ebb-ab64-746e3548a40f admin admin] PATH_INFO: `/v3/projects/ebf5f0daa032440ab81e5abda55f6e82/users/08661f6694ff44a0a4d15c792b54bb97/roles/df8f9f844b1b439181d75a07c1a93418` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:42.830200 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f894d1-3a54-4ebb-ab64-746e3548a40f admin admin] RBAC: Authorizing `identity:create_grant(project_id=ebf5f0daa032440ab81e5abda55f6e82, user_id=08661f6694ff44a0a4d15c792b54bb97, role_id=df8f9f844b1b439181d75a07c1a93418)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:42.832779 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f894d1-3a54-4ebb-ab64-746e3548a40f admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:42.849466 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1317/2609] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 30 16:05:42 2022] PUT /identity/v3/projects/ebf5f0daa032440ab81e5abda55f6e82/users/08661f6694ff44a0a4d15c792b54bb97/roles/df8f9f844b1b439181d75a07c1a93418 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:05:42.854928 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68ee3c2-32bd-4f48-95b6-55f884a5b884 None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:42.855315 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68ee3c2-32bd-4f48-95b6-55f884a5b884 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:42.855656 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68ee3c2-32bd-4f48-95b6-55f884a5b884 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:42.875165 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-a68ee3c2-32bd-4f48-95b6-55f884a5b884 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:05:42.876476 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-a68ee3c2-32bd-4f48-95b6-55f884a5b884 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:05:42.929201 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:42.941439 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1293/2610] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 16:05:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:05:42.948833 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f894d1-3a54-4ebb-ab64-746e3548a40f admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:42.950733 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-55f894d1-3a54-4ebb-ab64-746e3548a40f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:05:43.009887 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f894d1-3a54-4ebb-ab64-746e3548a40f admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:43.013613 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e432aff-e968-4983-a33c-2e00ace8c268 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f4157105f3488f807b7475008ab722', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:43.014233 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e432aff-e968-4983-a33c-2e00ace8c268 service neutron] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:43.014582 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e432aff-e968-4983-a33c-2e00ace8c268 service neutron] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:43.014766 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e432aff-e968-4983-a33c-2e00ace8c268 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:43.024253 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e432aff-e968-4983-a33c-2e00ace8c268 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:43.025734 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e432aff-e968-4983-a33c-2e00ace8c268 service neutron] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:43.035442 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1318/2611] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 16:05:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:05:43.927352 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3de13b0-f165-44a2-9476-1389bb5a0101 None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:43.927862 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3de13b0-f165-44a2-9476-1389bb5a0101 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:43.927862 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3de13b0-f165-44a2-9476-1389bb5a0101 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:43.967626 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-e3de13b0-f165-44a2-9476-1389bb5a0101 None None] MFA Rules not processed for user `08661f6694ff44a0a4d15c792b54bb97`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:05:43.969476 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-e3de13b0-f165-44a2-9476-1389bb5a0101 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:05:44.080821 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1294/2612] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 16:05:43 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:05:44.090453 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e432aff-e968-4983-a33c-2e00ace8c268 service neutron] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:44.091983 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-0e432aff-e968-4983-a33c-2e00ace8c268 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:05:44.176585 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e432aff-e968-4983-a33c-2e00ace8c268 service neutron] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:44.180752 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33695f93-62f2-43eb-bf77-fe79e524355b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0100a04821804993866ea20f159663f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:44.181477 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33695f93-62f2-43eb-bf77-fe79e524355b service cinder] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:44.181829 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33695f93-62f2-43eb-bf77-fe79e524355b service cinder] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:44.182170 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33695f93-62f2-43eb-bf77-fe79e524355b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:44.191572 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33695f93-62f2-43eb-bf77-fe79e524355b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:44.193092 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33695f93-62f2-43eb-bf77-fe79e524355b service cinder] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:44.203932 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1319/2613] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 16:05:44 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 16:05:45.211431 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3de13b0-f165-44a2-9476-1389bb5a0101 None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:45.212856 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-e3de13b0-f165-44a2-9476-1389bb5a0101 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:05:45.270542 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33695f93-62f2-43eb-bf77-fe79e524355b service cinder] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:45.272188 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-33695f93-62f2-43eb-bf77-fe79e524355b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:05:45.282878 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:45.291535 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33695f93-62f2-43eb-bf77-fe79e524355b service cinder] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:45.297855 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6a3822-b8c1-4bde-a730-78ace23c8d47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:45.298774 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6a3822-b8c1-4bde-a730-78ace23c8d47 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:45.299239 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6a3822-b8c1-4bde-a730-78ace23c8d47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:45.299512 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6a3822-b8c1-4bde-a730-78ace23c8d47 admin admin] PATH_INFO: `/v3/users/b133ae684f20468f982ae0c25020d837` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:45.300465 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3de13b0-f165-44a2-9476-1389bb5a0101 None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:45.300697 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6a3822-b8c1-4bde-a730-78ace23c8d47 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b133ae684f20468f982ae0c25020d837)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:45.303246 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6a3822-b8c1-4bde-a730-78ace23c8d47 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:45.306472 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ae0600-c1fb-46c4-a4f0-69301913ce3a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc12c6d31eab4a17a160250adc2380c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:45.307079 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ae0600-c1fb-46c4-a4f0-69301913ce3a service placement] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:45.307351 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ae0600-c1fb-46c4-a4f0-69301913ce3a service placement] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:45.307615 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ae0600-c1fb-46c4-a4f0-69301913ce3a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:45.309638 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-e9ae0600-c1fb-46c4-a4f0-69301913ce3a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:05:45.329870 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ae0600-c1fb-46c4-a4f0-69301913ce3a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:45.331533 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ae0600-c1fb-46c4-a4f0-69301913ce3a service placement] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:45.342390 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1295/2614] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 16:05:45 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:05:45.359450 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-4a6a3822-b8c1-4bde-a730-78ace23c8d47 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b133ae684f20468f982ae0c25020d837'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:05:45.374799 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-4a6a3822-b8c1-4bde-a730-78ace23c8d47 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b133ae684f20468f982ae0c25020d837'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:05:45.382976 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-4a6a3822-b8c1-4bde-a730-78ace23c8d47 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b133ae684f20468f982ae0c25020d837'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:05:45.383671 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-4a6a3822-b8c1-4bde-a730-78ace23c8d47 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b133ae684f20468f982ae0c25020d837'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:05:45.391820 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-4a6a3822-b8c1-4bde-a730-78ace23c8d47 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b133ae684f20468f982ae0c25020d837'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:05:45.398044 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1320/2615] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Aug 30 16:05:45 2022] DELETE /identity/v3/users/b133ae684f20468f982ae0c25020d837 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:05:45.577127 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ae0600-c1fb-46c4-a4f0-69301913ce3a service placement] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:45.589500 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ae0600-c1fb-46c4-a4f0-69301913ce3a service placement] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:45.620073 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854ee72d-526c-4826-a821-dc9e98839fcd None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:45.620356 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854ee72d-526c-4826-a821-dc9e98839fcd None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:45.620681 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854ee72d-526c-4826-a821-dc9e98839fcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:45.640381 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-854ee72d-526c-4826-a821-dc9e98839fcd None None] MFA Rules not processed for user `08661f6694ff44a0a4d15c792b54bb97`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:05:45.641706 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-854ee72d-526c-4826-a821-dc9e98839fcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:05:45.668602 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2603623-d1f0-4ba3-bbb3-633811546667 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:45.669250 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2603623-d1f0-4ba3-bbb3-633811546667 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:45.669531 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2603623-d1f0-4ba3-bbb3-633811546667 admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:45.669798 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2603623-d1f0-4ba3-bbb3-633811546667 admin admin] PATH_INFO: `/v3/projects/2c2aad2014bc4db49a818a6926f860d5` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:45.670900 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2603623-d1f0-4ba3-bbb3-633811546667 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2c2aad2014bc4db49a818a6926f860d5)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:45.672825 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2603623-d1f0-4ba3-bbb3-633811546667 admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:45.695979 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:45.698564 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1321/2616] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 16:05:45 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 16:05:45.721885 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-d2603623-d1f0-4ba3-bbb3-633811546667 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2c2aad2014bc4db49a818a6926f860d5', 'request_id': 'req-d2603623-d1f0-4ba3-bbb3-633811546667'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:05:45.730636 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-d2603623-d1f0-4ba3-bbb3-633811546667 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2c2aad2014bc4db49a818a6926f860d5', 'request_id': 'req-d2603623-d1f0-4ba3-bbb3-633811546667'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:05:45.738697 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-d2603623-d1f0-4ba3-bbb3-633811546667 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2c2aad2014bc4db49a818a6926f860d5', 'request_id': 'req-d2603623-d1f0-4ba3-bbb3-633811546667'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:05:45.749773 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-d2603623-d1f0-4ba3-bbb3-633811546667 admin admin] The token cache is being invalidate because project 2c2aad2014bc4db49a818a6926f860d5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81512) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 16:05:45.750066 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-d2603623-d1f0-4ba3-bbb3-633811546667 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:05:45.750745 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-d2603623-d1f0-4ba3-bbb3-633811546667 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:05:45.752229 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1296/2617] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Aug 30 16:05:45 2022] DELETE /identity/v3/projects/2c2aad2014bc4db49a818a6926f860d5 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 16:05:45.757151 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d15411-414d-4bf6-8236-050ca36c80ab None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:45.757469 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d15411-414d-4bf6-8236-050ca36c80ab None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:45.757739 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d15411-414d-4bf6-8236-050ca36c80ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:45.778660 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-e2d15411-414d-4bf6-8236-050ca36c80ab None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:05:45.780026 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-e2d15411-414d-4bf6-8236-050ca36c80ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:05:45.840646 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1322/2618] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 16:05:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:05:45.848347 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9918efd4-35dc-454f-b150-c63b11657fff None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:45.848635 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9918efd4-35dc-454f-b150-c63b11657fff None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:45.848913 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9918efd4-35dc-454f-b150-c63b11657fff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:45.869589 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-9918efd4-35dc-454f-b150-c63b11657fff None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:05:45.871133 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-9918efd4-35dc-454f-b150-c63b11657fff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:05:45.879279 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1297/2619] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 16:05:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:05:45.883568 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d15411-414d-4bf6-8236-050ca36c80ab None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:45.893421 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d15411-414d-4bf6-8236-050ca36c80ab None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:45.898140 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399a3057-7edf-4804-8674-64bcc3d36f4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:45.898751 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399a3057-7edf-4804-8674-64bcc3d36f4e admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:45.899029 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399a3057-7edf-4804-8674-64bcc3d36f4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:45.899299 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399a3057-7edf-4804-8674-64bcc3d36f4e admin admin] PATH_INFO: `/v3/domains` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:45.899953 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399a3057-7edf-4804-8674-64bcc3d36f4e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:05:45.900282 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399a3057-7edf-4804-8674-64bcc3d36f4e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:45.901935 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399a3057-7edf-4804-8674-64bcc3d36f4e admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:45.913958 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1323/2620] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Aug 30 16:05:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:05:45.918391 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9918efd4-35dc-454f-b150-c63b11657fff None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:45.927782 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9918efd4-35dc-454f-b150-c63b11657fff None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:45.932584 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59453b83-a564-4031-a705-dfe007b8c0f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:45.933191 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59453b83-a564-4031-a705-dfe007b8c0f1 admin admin] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:45.933499 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59453b83-a564-4031-a705-dfe007b8c0f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:45.933773 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59453b83-a564-4031-a705-dfe007b8c0f1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:45.934414 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59453b83-a564-4031-a705-dfe007b8c0f1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:45.936330 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59453b83-a564-4031-a705-dfe007b8c0f1 admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:45.940172 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399a3057-7edf-4804-8674-64bcc3d36f4e admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:45.942062 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-399a3057-7edf-4804-8674-64bcc3d36f4e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:05:45.951883 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:45.964836 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1298/2621] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Aug 30 16:05:45 2022] POST /identity/v3/projects => generated 371 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:05:45.969011 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59453b83-a564-4031-a705-dfe007b8c0f1 admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:46.007596 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399a3057-7edf-4804-8674-64bcc3d36f4e admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:46.030055 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59453b83-a564-4031-a705-dfe007b8c0f1 admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:46.034789 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b83fcb-d5be-4aae-aeed-cab781663392 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:46.035367 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b83fcb-d5be-4aae-aeed-cab781663392 admin admin] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:46.035646 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b83fcb-d5be-4aae-aeed-cab781663392 admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:46.035914 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b83fcb-d5be-4aae-aeed-cab781663392 admin admin] PATH_INFO: `/v3/users` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:46.036552 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b83fcb-d5be-4aae-aeed-cab781663392 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:46.038811 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b83fcb-d5be-4aae-aeed-cab781663392 admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:46.060866 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accc7687-09a8-4c78-8485-d77f1ac5517b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f4157105f3488f807b7475008ab722', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:46.061511 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accc7687-09a8-4c78-8485-d77f1ac5517b service neutron] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:46.061650 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1299/2622] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Aug 30 16:05:45 2022] POST /identity/v3/users => generated 286 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:05:46.064812 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accc7687-09a8-4c78-8485-d77f1ac5517b service neutron] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:46.065096 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accc7687-09a8-4c78-8485-d77f1ac5517b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:46.065700 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b83fcb-d5be-4aae-aeed-cab781663392 admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:46.066972 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-accc7687-09a8-4c78-8485-d77f1ac5517b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:05:46.076297 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b83fcb-d5be-4aae-aeed-cab781663392 admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:46.076647 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-accc7687-09a8-4c78-8485-d77f1ac5517b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:46.078119 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-accc7687-09a8-4c78-8485-d77f1ac5517b service neutron] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:46.080878 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938536b1-2ece-477d-9a80-217ca06e2e5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:46.081487 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938536b1-2ece-477d-9a80-217ca06e2e5b admin admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:46.081762 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938536b1-2ece-477d-9a80-217ca06e2e5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:46.082028 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938536b1-2ece-477d-9a80-217ca06e2e5b admin admin] PATH_INFO: `/v3/roles` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:46.082656 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938536b1-2ece-477d-9a80-217ca06e2e5b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:46.084244 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938536b1-2ece-477d-9a80-217ca06e2e5b admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:46.086549 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1324/2623] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 16:05:45 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 16:05:46.097537 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1300/2624] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 30 16:05:46 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 16:05:46.101409 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accc7687-09a8-4c78-8485-d77f1ac5517b service neutron] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:46.112668 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accc7687-09a8-4c78-8485-d77f1ac5517b service neutron] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:46.117499 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ad567b-5be1-4051-86fa-c354e5789974 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:46.118112 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ad567b-5be1-4051-86fa-c354e5789974 admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:46.118386 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ad567b-5be1-4051-86fa-c354e5789974 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:46.118656 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ad567b-5be1-4051-86fa-c354e5789974 admin admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:46.119299 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ad567b-5be1-4051-86fa-c354e5789974 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:46.120905 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ad567b-5be1-4051-86fa-c354e5789974 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:46.131300 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1325/2625] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 30 16:05:46 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 16:05:46.135696 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938536b1-2ece-477d-9a80-217ca06e2e5b admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:46.146089 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938536b1-2ece-477d-9a80-217ca06e2e5b admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:46.152162 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbed10f-4b5c-4282-a836-bd261be41c99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:46.152977 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbed10f-4b5c-4282-a836-bd261be41c99 admin admin] REQUEST_METHOD: `PUT` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:46.153464 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbed10f-4b5c-4282-a836-bd261be41c99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:46.153805 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbed10f-4b5c-4282-a836-bd261be41c99 admin admin] PATH_INFO: `/v3/projects/e181b24f6b4b41268657cd872cbc2849/users/2531da7730064bd5b082c1bb259d21eb/roles/df8f9f844b1b439181d75a07c1a93418` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:46.172001 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbed10f-4b5c-4282-a836-bd261be41c99 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e181b24f6b4b41268657cd872cbc2849, user_id=2531da7730064bd5b082c1bb259d21eb, role_id=df8f9f844b1b439181d75a07c1a93418)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:46.174777 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbed10f-4b5c-4282-a836-bd261be41c99 admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:46.190673 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1301/2626] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 30 16:05:46 2022] PUT /identity/v3/projects/e181b24f6b4b41268657cd872cbc2849/users/2531da7730064bd5b082c1bb259d21eb/roles/df8f9f844b1b439181d75a07c1a93418 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 16:05:46.201024 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5fa9b6-09f6-44ed-a9bb-eb8e12eafde0 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:46.201439 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5fa9b6-09f6-44ed-a9bb-eb8e12eafde0 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:46.201718 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5fa9b6-09f6-44ed-a9bb-eb8e12eafde0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:46.232069 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-7f5fa9b6-09f6-44ed-a9bb-eb8e12eafde0 None None] MFA Rules not processed for user `2531da7730064bd5b082c1bb259d21eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:05:46.233320 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-7f5fa9b6-09f6-44ed-a9bb-eb8e12eafde0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:05:46.304066 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1326/2627] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 16:05:46 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:05:46.316394 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbed10f-4b5c-4282-a836-bd261be41c99 admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:46.318744 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-1bbed10f-4b5c-4282-a836-bd261be41c99 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:05:46.382446 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:DELETE=7,keystone:SELECT=87,keystone:INSERT=6 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:46.430161 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbed10f-4b5c-4282-a836-bd261be41c99 admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:46.435817 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bede1e5-417a-4503-ac04-8df99b082e58 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a1e756ae9be4a7bac1e81acf07e8f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:46.436437 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bede1e5-417a-4503-ac04-8df99b082e58 service nova] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:46.436714 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bede1e5-417a-4503-ac04-8df99b082e58 service nova] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:46.436984 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bede1e5-417a-4503-ac04-8df99b082e58 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:46.446664 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bede1e5-417a-4503-ac04-8df99b082e58 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:46.448268 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bede1e5-417a-4503-ac04-8df99b082e58 service nova] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:46.465452 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1302/2628] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 16:05:46 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:05:49.225367 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5fa9b6-09f6-44ed-a9bb-eb8e12eafde0 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:49.295778 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:49.318042 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5fa9b6-09f6-44ed-a9bb-eb8e12eafde0 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:49.323068 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631dcd6c-16f6-410d-b789-b8cb1be7408d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:49.329378 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631dcd6c-16f6-410d-b789-b8cb1be7408d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:49.329742 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631dcd6c-16f6-410d-b789-b8cb1be7408d admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:49.330031 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631dcd6c-16f6-410d-b789-b8cb1be7408d admin admin] PATH_INFO: `/v3/users/2531da7730064bd5b082c1bb259d21eb` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:49.331361 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631dcd6c-16f6-410d-b789-b8cb1be7408d admin admin] RBAC: Authorizing `identity:delete_user(user_id=2531da7730064bd5b082c1bb259d21eb)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:49.333328 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631dcd6c-16f6-410d-b789-b8cb1be7408d admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:49.404847 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-631dcd6c-16f6-410d-b789-b8cb1be7408d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2531da7730064bd5b082c1bb259d21eb'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:05:49.417725 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-631dcd6c-16f6-410d-b789-b8cb1be7408d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2531da7730064bd5b082c1bb259d21eb'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:05:49.435699 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-631dcd6c-16f6-410d-b789-b8cb1be7408d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2531da7730064bd5b082c1bb259d21eb'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:05:49.436340 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-631dcd6c-16f6-410d-b789-b8cb1be7408d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2531da7730064bd5b082c1bb259d21eb'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:05:49.444806 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-631dcd6c-16f6-410d-b789-b8cb1be7408d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2531da7730064bd5b082c1bb259d21eb'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:05:49.451525 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1327/2629] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Aug 30 16:05:49 2022] DELETE /identity/v3/users/2531da7730064bd5b082c1bb259d21eb => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:05:49.456368 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837e85dd-acf7-40d8-a0c6-b904fbe76572 None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:49.456889 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837e85dd-acf7-40d8-a0c6-b904fbe76572 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:49.456988 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837e85dd-acf7-40d8-a0c6-b904fbe76572 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:49.475929 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-837e85dd-acf7-40d8-a0c6-b904fbe76572 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:05:49.477141 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-837e85dd-acf7-40d8-a0c6-b904fbe76572 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:05:49.543576 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1303/2630] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 16:05:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:05:49.553737 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631dcd6c-16f6-410d-b789-b8cb1be7408d admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:49.555277 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-631dcd6c-16f6-410d-b789-b8cb1be7408d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:05:49.618029 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631dcd6c-16f6-410d-b789-b8cb1be7408d admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:49.622193 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfae567-672c-4031-8bb9-b0d74a3c5e5d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f4157105f3488f807b7475008ab722', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:49.622815 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfae567-672c-4031-8bb9-b0d74a3c5e5d service neutron] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:49.623089 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfae567-672c-4031-8bb9-b0d74a3c5e5d service neutron] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:49.623359 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfae567-672c-4031-8bb9-b0d74a3c5e5d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:49.637021 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bfae567-672c-4031-8bb9-b0d74a3c5e5d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:49.639395 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bfae567-672c-4031-8bb9-b0d74a3c5e5d service neutron] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:49.650369 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1328/2631] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 16:05:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:05:49.981993 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837e85dd-acf7-40d8-a0c6-b904fbe76572 None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:49.983548 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-837e85dd-acf7-40d8-a0c6-b904fbe76572 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:05:49.996459 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837e85dd-acf7-40d8-a0c6-b904fbe76572 None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:50.001283 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c95590-af55-408c-8dda-3444cfa5b569 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:50.001922 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c95590-af55-408c-8dda-3444cfa5b569 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:50.002225 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c95590-af55-408c-8dda-3444cfa5b569 admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:50.002490 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c95590-af55-408c-8dda-3444cfa5b569 admin admin] PATH_INFO: `/v3/projects/e181b24f6b4b41268657cd872cbc2849` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:50.003658 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c95590-af55-408c-8dda-3444cfa5b569 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e181b24f6b4b41268657cd872cbc2849)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:50.005673 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c95590-af55-408c-8dda-3444cfa5b569 admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:50.055630 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-54c95590-af55-408c-8dda-3444cfa5b569 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e181b24f6b4b41268657cd872cbc2849', 'request_id': 'req-54c95590-af55-408c-8dda-3444cfa5b569'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:05:50.064337 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-54c95590-af55-408c-8dda-3444cfa5b569 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e181b24f6b4b41268657cd872cbc2849', 'request_id': 'req-54c95590-af55-408c-8dda-3444cfa5b569'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:05:50.070634 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-54c95590-af55-408c-8dda-3444cfa5b569 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e181b24f6b4b41268657cd872cbc2849', 'request_id': 'req-54c95590-af55-408c-8dda-3444cfa5b569'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:05:50.081205 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-54c95590-af55-408c-8dda-3444cfa5b569 admin admin] The token cache is being invalidate because project e181b24f6b4b41268657cd872cbc2849 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81512) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 16:05:50.081538 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-54c95590-af55-408c-8dda-3444cfa5b569 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:05:50.082195 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-54c95590-af55-408c-8dda-3444cfa5b569 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:05:50.083728 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1304/2632] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Aug 30 16:05:49 2022] DELETE /identity/v3/projects/e181b24f6b4b41268657cd872cbc2849 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:05:50.088831 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4b3589-38a4-436e-b841-6a75e933db0c None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:50.089116 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4b3589-38a4-436e-b841-6a75e933db0c None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:50.089425 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4b3589-38a4-436e-b841-6a75e933db0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:50.109861 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-0f4b3589-38a4-436e-b841-6a75e933db0c None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:05:50.111101 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-0f4b3589-38a4-436e-b841-6a75e933db0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:05:50.174121 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1329/2633] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 16:05:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:05:50.183222 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4b0ef2-bc49-4b65-a1d3-92a3d272ac1d None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:50.183526 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4b0ef2-bc49-4b65-a1d3-92a3d272ac1d None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:50.183782 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4b0ef2-bc49-4b65-a1d3-92a3d272ac1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:50.205655 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-eb4b0ef2-bc49-4b65-a1d3-92a3d272ac1d None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:05:50.206867 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-eb4b0ef2-bc49-4b65-a1d3-92a3d272ac1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:05:50.215520 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1305/2634] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 16:05:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:05:50.219959 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4b3589-38a4-436e-b841-6a75e933db0c None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:50.229914 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4b3589-38a4-436e-b841-6a75e933db0c None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:50.235322 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da862f2f-229a-4aab-b6fc-9fbf70dae2d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:50.235930 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da862f2f-229a-4aab-b6fc-9fbf70dae2d4 admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:50.236207 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da862f2f-229a-4aab-b6fc-9fbf70dae2d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:50.236473 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da862f2f-229a-4aab-b6fc-9fbf70dae2d4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:50.237098 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da862f2f-229a-4aab-b6fc-9fbf70dae2d4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:05:50.237637 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da862f2f-229a-4aab-b6fc-9fbf70dae2d4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:50.239288 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da862f2f-229a-4aab-b6fc-9fbf70dae2d4 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:50.240941 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:50.255572 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1330/2635] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Aug 30 16:05:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:05:50.260169 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4b0ef2-bc49-4b65-a1d3-92a3d272ac1d None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:50.269648 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4b0ef2-bc49-4b65-a1d3-92a3d272ac1d None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:50.274737 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f084c8-3d74-4ddb-ba04-64f7c9df3875 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:50.275332 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f084c8-3d74-4ddb-ba04-64f7c9df3875 admin admin] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:50.275604 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f084c8-3d74-4ddb-ba04-64f7c9df3875 admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:50.275867 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f084c8-3d74-4ddb-ba04-64f7c9df3875 admin admin] PATH_INFO: `/v3/projects` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:50.276494 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f084c8-3d74-4ddb-ba04-64f7c9df3875 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:50.278433 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f084c8-3d74-4ddb-ba04-64f7c9df3875 admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:50.308005 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1306/2636] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Aug 30 16:05:50 2022] POST /identity/v3/projects => generated 365 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:05:50.312219 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da862f2f-229a-4aab-b6fc-9fbf70dae2d4 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:50.322533 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da862f2f-229a-4aab-b6fc-9fbf70dae2d4 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:50.381012 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50827dd0-c8b6-402d-b2c3-222aa1ff9da0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:50.381654 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50827dd0-c8b6-402d-b2c3-222aa1ff9da0 admin admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:50.381975 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50827dd0-c8b6-402d-b2c3-222aa1ff9da0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:50.382243 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50827dd0-c8b6-402d-b2c3-222aa1ff9da0 admin admin] PATH_INFO: `/v3/users` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:50.382864 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50827dd0-c8b6-402d-b2c3-222aa1ff9da0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:50.384791 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50827dd0-c8b6-402d-b2c3-222aa1ff9da0 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:50.408252 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1331/2637] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Aug 30 16:05:50 2022] POST /identity/v3/users => generated 283 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:05:50.412527 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f084c8-3d74-4ddb-ba04-64f7c9df3875 admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:50.423152 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f084c8-3d74-4ddb-ba04-64f7c9df3875 admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:50.428218 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb73c284-8ded-40bf-a185-fba2c460bd5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:50.428840 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb73c284-8ded-40bf-a185-fba2c460bd5e admin admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:50.429466 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb73c284-8ded-40bf-a185-fba2c460bd5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:50.429466 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb73c284-8ded-40bf-a185-fba2c460bd5e admin admin] PATH_INFO: `/v3/roles` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:50.430056 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb73c284-8ded-40bf-a185-fba2c460bd5e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:50.431666 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb73c284-8ded-40bf-a185-fba2c460bd5e admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:50.442046 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1307/2638] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 30 16:05:50 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:05:50.446148 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50827dd0-c8b6-402d-b2c3-222aa1ff9da0 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:50.456136 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50827dd0-c8b6-402d-b2c3-222aa1ff9da0 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:50.461009 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f8a454-99f9-4bdb-9c57-b2a905218da5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:50.461637 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f8a454-99f9-4bdb-9c57-b2a905218da5 admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:50.461914 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f8a454-99f9-4bdb-9c57-b2a905218da5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:50.462181 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f8a454-99f9-4bdb-9c57-b2a905218da5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:50.462819 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f8a454-99f9-4bdb-9c57-b2a905218da5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:50.464436 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f8a454-99f9-4bdb-9c57-b2a905218da5 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:50.475134 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1332/2639] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 30 16:05:50 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:05:50.479473 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb73c284-8ded-40bf-a185-fba2c460bd5e admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:50.486281 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:50.490381 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb73c284-8ded-40bf-a185-fba2c460bd5e admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:50.497172 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca18e8e6-5071-4639-83d8-6b541136ab44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:50.497913 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca18e8e6-5071-4639-83d8-6b541136ab44 admin admin] REQUEST_METHOD: `PUT` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:50.498191 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca18e8e6-5071-4639-83d8-6b541136ab44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:50.498468 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca18e8e6-5071-4639-83d8-6b541136ab44 admin admin] PATH_INFO: `/v3/projects/6029157c77e14de3a080a948e534013f/users/8ba65da714cf4744b8d90494e60cac57/roles/df8f9f844b1b439181d75a07c1a93418` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:50.514189 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca18e8e6-5071-4639-83d8-6b541136ab44 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6029157c77e14de3a080a948e534013f, user_id=8ba65da714cf4744b8d90494e60cac57, role_id=df8f9f844b1b439181d75a07c1a93418)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:50.516832 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca18e8e6-5071-4639-83d8-6b541136ab44 admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:50.532563 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1308/2640] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 30 16:05:50 2022] PUT /identity/v3/projects/6029157c77e14de3a080a948e534013f/users/8ba65da714cf4744b8d90494e60cac57/roles/df8f9f844b1b439181d75a07c1a93418 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:05:50.543376 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743dfdf4-b4f5-4c8b-8d2e-e5ea791ec960 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:50.543665 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743dfdf4-b4f5-4c8b-8d2e-e5ea791ec960 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:50.543945 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743dfdf4-b4f5-4c8b-8d2e-e5ea791ec960 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:50.572228 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-743dfdf4-b4f5-4c8b-8d2e-e5ea791ec960 None None] MFA Rules not processed for user `8ba65da714cf4744b8d90494e60cac57`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:05:50.573481 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-743dfdf4-b4f5-4c8b-8d2e-e5ea791ec960 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:05:50.649761 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1333/2641] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 16:05:50 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:05:50.664544 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca18e8e6-5071-4639-83d8-6b541136ab44 admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:50.666647 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-ca18e8e6-5071-4639-83d8-6b541136ab44 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:05:50.727563 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca18e8e6-5071-4639-83d8-6b541136ab44 admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:50.733406 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff502c1-0c6e-4b40-b35c-b72c76832f81 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0100a04821804993866ea20f159663f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:50.734021 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff502c1-0c6e-4b40-b35c-b72c76832f81 service cinder] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:50.734301 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff502c1-0c6e-4b40-b35c-b72c76832f81 service cinder] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:50.734572 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff502c1-0c6e-4b40-b35c-b72c76832f81 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:50.743214 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff502c1-0c6e-4b40-b35c-b72c76832f81 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:50.744708 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff502c1-0c6e-4b40-b35c-b72c76832f81 service cinder] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:50.753435 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1309/2642] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 16:05:50 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:05:59.275626 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743dfdf4-b4f5-4c8b-8d2e-e5ea791ec960 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:59.358091 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743dfdf4-b4f5-4c8b-8d2e-e5ea791ec960 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:59.364041 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d81f7e-ff10-4815-b316-cbc00129dbaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:59.364866 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d81f7e-ff10-4815-b316-cbc00129dbaa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:59.365139 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d81f7e-ff10-4815-b316-cbc00129dbaa admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:59.365726 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d81f7e-ff10-4815-b316-cbc00129dbaa admin admin] PATH_INFO: `/v3/users/8ba65da714cf4744b8d90494e60cac57` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:59.367118 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d81f7e-ff10-4815-b316-cbc00129dbaa admin admin] RBAC: Authorizing `identity:delete_user(user_id=8ba65da714cf4744b8d90494e60cac57)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:59.369079 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d81f7e-ff10-4815-b316-cbc00129dbaa admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:59.371874 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:59.422806 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-84d81f7e-ff10-4815-b316-cbc00129dbaa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8ba65da714cf4744b8d90494e60cac57'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:05:59.435636 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-84d81f7e-ff10-4815-b316-cbc00129dbaa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8ba65da714cf4744b8d90494e60cac57'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:05:59.444985 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-84d81f7e-ff10-4815-b316-cbc00129dbaa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8ba65da714cf4744b8d90494e60cac57'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:05:59.445721 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-84d81f7e-ff10-4815-b316-cbc00129dbaa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8ba65da714cf4744b8d90494e60cac57'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:05:59.453322 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-84d81f7e-ff10-4815-b316-cbc00129dbaa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8ba65da714cf4744b8d90494e60cac57'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:05:59.459461 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1334/2643] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Aug 30 16:05:59 2022] DELETE /identity/v3/users/8ba65da714cf4744b8d90494e60cac57 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:05:59.464068 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe013198-481b-463b-b928-882740e928c2 None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:59.464567 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe013198-481b-463b-b928-882740e928c2 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:59.464567 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe013198-481b-463b-b928-882740e928c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:59.484676 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-fe013198-481b-463b-b928-882740e928c2 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:05:59.485937 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-fe013198-481b-463b-b928-882740e928c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:05:59.555678 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1310/2644] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 16:05:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:05:59.566439 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d81f7e-ff10-4815-b316-cbc00129dbaa admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:59.567943 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-84d81f7e-ff10-4815-b316-cbc00129dbaa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:05:59.634258 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d81f7e-ff10-4815-b316-cbc00129dbaa admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:59.638473 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13554107-83fe-4e63-b9a2-ab562132b653 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f4157105f3488f807b7475008ab722', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:59.639200 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13554107-83fe-4e63-b9a2-ab562132b653 service neutron] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:59.639891 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13554107-83fe-4e63-b9a2-ab562132b653 service neutron] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:59.640161 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13554107-83fe-4e63-b9a2-ab562132b653 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:59.651506 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13554107-83fe-4e63-b9a2-ab562132b653 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:59.653399 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13554107-83fe-4e63-b9a2-ab562132b653 service neutron] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:59.665443 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1335/2645] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 16:05:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:05:59.951837 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe013198-481b-463b-b928-882740e928c2 None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:59.953670 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-fe013198-481b-463b-b928-882740e928c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:05:59.967101 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe013198-481b-463b-b928-882740e928c2 None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:59.974673 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc60625-aed9-44f4-85bb-4dd22d3c6c9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:59.974897 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc60625-aed9-44f4-85bb-4dd22d3c6c9b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:59.975057 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc60625-aed9-44f4-85bb-4dd22d3c6c9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:59.975334 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc60625-aed9-44f4-85bb-4dd22d3c6c9b admin admin] PATH_INFO: `/v3/projects/6029157c77e14de3a080a948e534013f` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:59.976578 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc60625-aed9-44f4-85bb-4dd22d3c6c9b admin admin] RBAC: Authorizing `identity:delete_project(project_id=6029157c77e14de3a080a948e534013f)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:59.978685 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc60625-aed9-44f4-85bb-4dd22d3c6c9b admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:06:00.033258 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-bdc60625-aed9-44f4-85bb-4dd22d3c6c9b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6029157c77e14de3a080a948e534013f', 'request_id': 'req-bdc60625-aed9-44f4-85bb-4dd22d3c6c9b'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:06:00.042949 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-bdc60625-aed9-44f4-85bb-4dd22d3c6c9b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6029157c77e14de3a080a948e534013f', 'request_id': 'req-bdc60625-aed9-44f4-85bb-4dd22d3c6c9b'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:06:00.049800 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-bdc60625-aed9-44f4-85bb-4dd22d3c6c9b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6029157c77e14de3a080a948e534013f', 'request_id': 'req-bdc60625-aed9-44f4-85bb-4dd22d3c6c9b'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:06:00.062356 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-bdc60625-aed9-44f4-85bb-4dd22d3c6c9b admin admin] The token cache is being invalidate because project 6029157c77e14de3a080a948e534013f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81512) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 16:06:00.062659 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-bdc60625-aed9-44f4-85bb-4dd22d3c6c9b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:06:00.063435 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-bdc60625-aed9-44f4-85bb-4dd22d3c6c9b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:06:00.065085 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1311/2646] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Aug 30 16:05:59 2022] DELETE /identity/v3/projects/6029157c77e14de3a080a948e534013f => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 16:06:00.071635 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b06bdc-bdb4-49ef-a5a2-5cf5559134fe None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:06:00.071858 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b06bdc-bdb4-49ef-a5a2-5cf5559134fe None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:06:00.072260 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b06bdc-bdb4-49ef-a5a2-5cf5559134fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:06:00.093017 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-a7b06bdc-bdb4-49ef-a5a2-5cf5559134fe None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:06:00.094259 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-a7b06bdc-bdb4-49ef-a5a2-5cf5559134fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:06:00.160326 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1336/2647] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 16:06:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:06:00.169343 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ba754c-4695-4736-8f35-3487d59e9766 None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:06:00.169636 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ba754c-4695-4736-8f35-3487d59e9766 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:06:00.169910 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ba754c-4695-4736-8f35-3487d59e9766 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:06:00.189297 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:06:00.196579 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-24ba754c-4695-4736-8f35-3487d59e9766 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:06:00.199024 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-24ba754c-4695-4736-8f35-3487d59e9766 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:06:00.208195 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1312/2648] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 16:06:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:06:00.213005 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b06bdc-bdb4-49ef-a5a2-5cf5559134fe None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:06:00.224221 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b06bdc-bdb4-49ef-a5a2-5cf5559134fe None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:06:00.229394 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed230f3d-e8db-4306-bc51-e6dd2a258eef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:06:00.230012 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed230f3d-e8db-4306-bc51-e6dd2a258eef admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:06:00.230311 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed230f3d-e8db-4306-bc51-e6dd2a258eef admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:06:00.230572 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed230f3d-e8db-4306-bc51-e6dd2a258eef admin admin] PATH_INFO: `/v3/domains` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:06:00.231579 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed230f3d-e8db-4306-bc51-e6dd2a258eef admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:06:00.231878 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed230f3d-e8db-4306-bc51-e6dd2a258eef admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:06:00.233515 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed230f3d-e8db-4306-bc51-e6dd2a258eef admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:06:00.246548 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1337/2649] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Aug 30 16:06:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:06:00.251222 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ba754c-4695-4736-8f35-3487d59e9766 None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:06:00.260758 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ba754c-4695-4736-8f35-3487d59e9766 None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:06:00.265862 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09648c1a-a37e-4799-87f6-0d7d0867e40d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:06:00.266456 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09648c1a-a37e-4799-87f6-0d7d0867e40d admin admin] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:06:00.266731 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09648c1a-a37e-4799-87f6-0d7d0867e40d admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:06:00.267003 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09648c1a-a37e-4799-87f6-0d7d0867e40d admin admin] PATH_INFO: `/v3/projects` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:06:00.267633 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09648c1a-a37e-4799-87f6-0d7d0867e40d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:06:00.269937 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09648c1a-a37e-4799-87f6-0d7d0867e40d admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:06:00.302893 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1313/2650] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Aug 30 16:06:00 2022] POST /identity/v3/projects => generated 405 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:06:00.308945 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed230f3d-e8db-4306-bc51-e6dd2a258eef admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:06:00.319622 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed230f3d-e8db-4306-bc51-e6dd2a258eef admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:06:00.329283 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:06:00.382575 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a8fc0a-88af-499e-99bd-4abefd35a04b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:06:00.383431 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a8fc0a-88af-499e-99bd-4abefd35a04b admin admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:06:00.383769 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a8fc0a-88af-499e-99bd-4abefd35a04b admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:06:00.384062 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a8fc0a-88af-499e-99bd-4abefd35a04b admin admin] PATH_INFO: `/v3/users` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:06:00.384677 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a8fc0a-88af-499e-99bd-4abefd35a04b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:06:00.387013 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a8fc0a-88af-499e-99bd-4abefd35a04b admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:06:00.411679 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1338/2651] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Aug 30 16:06:00 2022] POST /identity/v3/users => generated 303 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:06:00.416293 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09648c1a-a37e-4799-87f6-0d7d0867e40d admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:06:00.426753 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09648c1a-a37e-4799-87f6-0d7d0867e40d admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:06:00.431441 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f44a173-c67b-4e9d-9129-447b49d0be95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:06:00.432043 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f44a173-c67b-4e9d-9129-447b49d0be95 admin admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:06:00.432317 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f44a173-c67b-4e9d-9129-447b49d0be95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:06:00.432584 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f44a173-c67b-4e9d-9129-447b49d0be95 admin admin] PATH_INFO: `/v3/roles` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:06:00.433270 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f44a173-c67b-4e9d-9129-447b49d0be95 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:06:00.434922 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f44a173-c67b-4e9d-9129-447b49d0be95 admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:06:00.445930 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1314/2652] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 30 16:06:00 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:06:00.452325 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a8fc0a-88af-499e-99bd-4abefd35a04b admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:06:00.467139 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a8fc0a-88af-499e-99bd-4abefd35a04b admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:06:00.472980 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c00f843-2d22-475c-84ef-20b806de008e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:06:00.473631 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c00f843-2d22-475c-84ef-20b806de008e admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:06:00.473970 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c00f843-2d22-475c-84ef-20b806de008e admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:06:00.474356 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c00f843-2d22-475c-84ef-20b806de008e admin admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:06:00.475061 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c00f843-2d22-475c-84ef-20b806de008e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:06:00.476804 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c00f843-2d22-475c-84ef-20b806de008e admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:06:00.489381 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1339/2653] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 30 16:06:00 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:06:00.493848 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f44a173-c67b-4e9d-9129-447b49d0be95 admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:06:00.504917 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f44a173-c67b-4e9d-9129-447b49d0be95 admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:06:00.511375 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36936beb-8dc2-4363-9f03-b4950303d160 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:06:00.512053 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36936beb-8dc2-4363-9f03-b4950303d160 admin admin] REQUEST_METHOD: `PUT` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:06:00.512329 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36936beb-8dc2-4363-9f03-b4950303d160 admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:06:00.512598 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36936beb-8dc2-4363-9f03-b4950303d160 admin admin] PATH_INFO: `/v3/projects/00e538489fc046ae86fd1dcb836a51b7/users/3cfa94c01a2342aea116948357f3813c/roles/df8f9f844b1b439181d75a07c1a93418` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:06:00.528403 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36936beb-8dc2-4363-9f03-b4950303d160 admin admin] RBAC: Authorizing `identity:create_grant(project_id=00e538489fc046ae86fd1dcb836a51b7, user_id=3cfa94c01a2342aea116948357f3813c, role_id=df8f9f844b1b439181d75a07c1a93418)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:06:00.531170 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36936beb-8dc2-4363-9f03-b4950303d160 admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:06:00.547385 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1315/2654] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 30 16:06:00 2022] PUT /identity/v3/projects/00e538489fc046ae86fd1dcb836a51b7/users/3cfa94c01a2342aea116948357f3813c/roles/df8f9f844b1b439181d75a07c1a93418 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:06:00.559584 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945d792d-eac1-4dfe-9e5a-25dc73f16172 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:06:00.559878 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945d792d-eac1-4dfe-9e5a-25dc73f16172 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:06:00.560141 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945d792d-eac1-4dfe-9e5a-25dc73f16172 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:06:00.589481 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-945d792d-eac1-4dfe-9e5a-25dc73f16172 None None] MFA Rules not processed for user `3cfa94c01a2342aea116948357f3813c`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:06:00.590659 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-945d792d-eac1-4dfe-9e5a-25dc73f16172 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:06:00.659963 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1340/2655] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 16:06:00 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:06:00.664720 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36936beb-8dc2-4363-9f03-b4950303d160 admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:06:00.731788 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36936beb-8dc2-4363-9f03-b4950303d160 admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:06:00.736691 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63c6ae5-cd08-4a5c-9ffd-d70bb185d567 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:06:00.737320 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63c6ae5-cd08-4a5c-9ffd-d70bb185d567 admin admin] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:06:00.737596 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63c6ae5-cd08-4a5c-9ffd-d70bb185d567 admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:06:00.737863 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63c6ae5-cd08-4a5c-9ffd-d70bb185d567 admin admin] PATH_INFO: `/v3/projects` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:06:00.738486 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63c6ae5-cd08-4a5c-9ffd-d70bb185d567 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:06:00.740380 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63c6ae5-cd08-4a5c-9ffd-d70bb185d567 admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:06:00.771451 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1316/2656] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Aug 30 16:06:00 2022] POST /identity/v3/projects => generated 405 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 16:06:00.775645 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945d792d-eac1-4dfe-9e5a-25dc73f16172 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:06:00.786904 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945d792d-eac1-4dfe-9e5a-25dc73f16172 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:06:00.847342 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05283d12-cf0d-49fa-ac48-0d93317a05ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:06:00.847897 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05283d12-cf0d-49fa-ac48-0d93317a05ac admin admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:06:00.848173 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05283d12-cf0d-49fa-ac48-0d93317a05ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:06:00.848437 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05283d12-cf0d-49fa-ac48-0d93317a05ac admin admin] PATH_INFO: `/v3/users` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:06:00.849106 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05283d12-cf0d-49fa-ac48-0d93317a05ac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:06:00.851090 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05283d12-cf0d-49fa-ac48-0d93317a05ac admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:06:00.874654 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1341/2657] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Aug 30 16:06:00 2022] POST /identity/v3/users => generated 309 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:06:00.878893 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63c6ae5-cd08-4a5c-9ffd-d70bb185d567 admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:06:00.889340 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63c6ae5-cd08-4a5c-9ffd-d70bb185d567 admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:06:00.894154 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17118097-73a4-41fe-a551-908dde5f2c42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:06:00.894739 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17118097-73a4-41fe-a551-908dde5f2c42 admin admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:06:00.895014 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17118097-73a4-41fe-a551-908dde5f2c42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:06:00.895282 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17118097-73a4-41fe-a551-908dde5f2c42 admin admin] PATH_INFO: `/v3/roles` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:06:00.895920 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17118097-73a4-41fe-a551-908dde5f2c42 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:06:00.897590 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17118097-73a4-41fe-a551-908dde5f2c42 admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:06:00.909102 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1317/2658] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 30 16:06:00 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:06:00.913680 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05283d12-cf0d-49fa-ac48-0d93317a05ac admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:06:00.924742 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05283d12-cf0d-49fa-ac48-0d93317a05ac admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:06:00.930282 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec40031-34c4-4122-b743-430ad078c5f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:06:00.930989 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec40031-34c4-4122-b743-430ad078c5f8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:06:00.931269 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec40031-34c4-4122-b743-430ad078c5f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:06:00.931550 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec40031-34c4-4122-b743-430ad078c5f8 admin admin] PATH_INFO: `/v3/projects/5570b36de34b4ff8b4612eda43aa8df1/users/8961468ecdd241c4920acbbb98401c11/roles/f265386950a641ac8413b53eda4ae34d` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:06:00.934890 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:06:00.947817 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec40031-34c4-4122-b743-430ad078c5f8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5570b36de34b4ff8b4612eda43aa8df1, user_id=8961468ecdd241c4920acbbb98401c11, role_id=f265386950a641ac8413b53eda4ae34d)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:06:00.950632 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec40031-34c4-4122-b743-430ad078c5f8 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:06:00.968632 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1342/2659] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 30 16:06:00 2022] PUT /identity/v3/projects/5570b36de34b4ff8b4612eda43aa8df1/users/8961468ecdd241c4920acbbb98401c11/roles/f265386950a641ac8413b53eda4ae34d => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:06:00.972882 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17118097-73a4-41fe-a551-908dde5f2c42 admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:06:00.983971 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17118097-73a4-41fe-a551-908dde5f2c42 admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:06:01.043734 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0d5ff1-39fc-438d-9b55-6519b23b44ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:06:01.044373 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0d5ff1-39fc-438d-9b55-6519b23b44ce admin admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:06:01.044653 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0d5ff1-39fc-438d-9b55-6519b23b44ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:06:01.044923 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0d5ff1-39fc-438d-9b55-6519b23b44ce admin admin] PATH_INFO: `/v3/roles` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:06:01.045609 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0d5ff1-39fc-438d-9b55-6519b23b44ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:06:01.047219 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0d5ff1-39fc-438d-9b55-6519b23b44ce admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:06:01.057936 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1318/2660] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 30 16:06:00 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:06:01.062088 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec40031-34c4-4122-b743-430ad078c5f8 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:06:01.072983 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec40031-34c4-4122-b743-430ad078c5f8 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:06:01.077686 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8145fb-3b2b-4bcc-aed5-141428095ed1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:06:01.078337 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8145fb-3b2b-4bcc-aed5-141428095ed1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:06:01.078638 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8145fb-3b2b-4bcc-aed5-141428095ed1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:06:01.078882 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8145fb-3b2b-4bcc-aed5-141428095ed1 admin admin] PATH_INFO: `/v3/projects/5570b36de34b4ff8b4612eda43aa8df1/users/8961468ecdd241c4920acbbb98401c11/roles/f265386950a641ac8413b53eda4ae34d` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:06:01.080826 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8145fb-3b2b-4bcc-aed5-141428095ed1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5570b36de34b4ff8b4612eda43aa8df1, user_id=8961468ecdd241c4920acbbb98401c11, role_id=f265386950a641ac8413b53eda4ae34d)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:06:01.083475 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8145fb-3b2b-4bcc-aed5-141428095ed1 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:06:01.094459 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1343/2661] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 30 16:06:01 2022] PUT /identity/v3/projects/5570b36de34b4ff8b4612eda43aa8df1/users/8961468ecdd241c4920acbbb98401c11/roles/f265386950a641ac8413b53eda4ae34d => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:06:01.104898 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6a8540-530b-40c2-a16b-ba9b1596e6e8 None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:06:01.105074 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6a8540-530b-40c2-a16b-ba9b1596e6e8 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:06:01.105439 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6a8540-530b-40c2-a16b-ba9b1596e6e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:06:01.132842 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:06:01.136731 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-df6a8540-530b-40c2-a16b-ba9b1596e6e8 None None] MFA Rules not processed for user `8961468ecdd241c4920acbbb98401c11`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:06:01.138814 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-df6a8540-530b-40c2-a16b-ba9b1596e6e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:06:01.211178 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1319/2662] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 16:06:01 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:06:01.223306 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8145fb-3b2b-4bcc-aed5-141428095ed1 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:06:01.225064 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-4e8145fb-3b2b-4bcc-aed5-141428095ed1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:06:01.290739 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8145fb-3b2b-4bcc-aed5-141428095ed1 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:06:01.294553 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7e38bd-6a72-4d9e-acdf-f75971862e66 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0100a04821804993866ea20f159663f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:06:01.295549 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7e38bd-6a72-4d9e-acdf-f75971862e66 service cinder] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:06:01.296033 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7e38bd-6a72-4d9e-acdf-f75971862e66 service cinder] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:06:01.296488 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7e38bd-6a72-4d9e-acdf-f75971862e66 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:06:01.308417 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7e38bd-6a72-4d9e-acdf-f75971862e66 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:06:01.310005 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7e38bd-6a72-4d9e-acdf-f75971862e66 service cinder] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:06:01.322358 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1344/2663] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 16:06:01 2022] GET /identity/v3/auth/tokens => generated 3349 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:06:01.486015 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6a8540-530b-40c2-a16b-ba9b1596e6e8 None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:06:01.554979 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6a8540-530b-40c2-a16b-ba9b1596e6e8 None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:06:01.559773 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129f7379-97a0-4a7e-8dca-8ee5963b9b41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:06:01.560409 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129f7379-97a0-4a7e-8dca-8ee5963b9b41 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:06:01.560686 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129f7379-97a0-4a7e-8dca-8ee5963b9b41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:06:01.560968 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129f7379-97a0-4a7e-8dca-8ee5963b9b41 admin admin] PATH_INFO: `/v3/users/3cfa94c01a2342aea116948357f3813c` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:06:01.562239 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129f7379-97a0-4a7e-8dca-8ee5963b9b41 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3cfa94c01a2342aea116948357f3813c)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:06:01.564170 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129f7379-97a0-4a7e-8dca-8ee5963b9b41 admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:06:01.625512 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-129f7379-97a0-4a7e-8dca-8ee5963b9b41 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3cfa94c01a2342aea116948357f3813c'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:06:01.635757 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-129f7379-97a0-4a7e-8dca-8ee5963b9b41 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3cfa94c01a2342aea116948357f3813c'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:06:01.651378 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-129f7379-97a0-4a7e-8dca-8ee5963b9b41 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3cfa94c01a2342aea116948357f3813c'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:06:01.658372 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-129f7379-97a0-4a7e-8dca-8ee5963b9b41 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3cfa94c01a2342aea116948357f3813c'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:06:01.659249 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-129f7379-97a0-4a7e-8dca-8ee5963b9b41 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3cfa94c01a2342aea116948357f3813c'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:06:01.665380 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1320/2664] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Aug 30 16:06:01 2022] DELETE /identity/v3/users/3cfa94c01a2342aea116948357f3813c => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:06:01.670284 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a8ea4e-2730-43bc-9e0d-5ca509a4162c None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:06:01.670564 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a8ea4e-2730-43bc-9e0d-5ca509a4162c None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:06:01.670883 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a8ea4e-2730-43bc-9e0d-5ca509a4162c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:06:01.692679 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-d8a8ea4e-2730-43bc-9e0d-5ca509a4162c None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:06:01.693952 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-d8a8ea4e-2730-43bc-9e0d-5ca509a4162c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:06:01.889889 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1345/2665] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 16:06:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:06:01.896518 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129f7379-97a0-4a7e-8dca-8ee5963b9b41 admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:06:01.898105 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-129f7379-97a0-4a7e-8dca-8ee5963b9b41 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:06:01.927937 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:06:01.969743 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129f7379-97a0-4a7e-8dca-8ee5963b9b41 admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:06:01.973513 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba632bd-baab-4455-8914-d29a70dfcdfe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f4157105f3488f807b7475008ab722', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:06:01.974116 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba632bd-baab-4455-8914-d29a70dfcdfe service neutron] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:06:01.974395 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba632bd-baab-4455-8914-d29a70dfcdfe service neutron] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:06:01.974661 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba632bd-baab-4455-8914-d29a70dfcdfe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:06:01.985653 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba632bd-baab-4455-8914-d29a70dfcdfe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:06:01.987116 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba632bd-baab-4455-8914-d29a70dfcdfe service neutron] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:06:01.997749 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1321/2666] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 16:06:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:06:02.296274 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a8ea4e-2730-43bc-9e0d-5ca509a4162c None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:06:02.298189 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-d8a8ea4e-2730-43bc-9e0d-5ca509a4162c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:06:02.311591 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a8ea4e-2730-43bc-9e0d-5ca509a4162c None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:06:02.317567 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7128689a-8574-4c56-820d-0ce55d773b6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:06:02.318283 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7128689a-8574-4c56-820d-0ce55d773b6e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:06:02.318621 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7128689a-8574-4c56-820d-0ce55d773b6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:06:02.318951 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7128689a-8574-4c56-820d-0ce55d773b6e admin admin] PATH_INFO: `/v3/projects/00e538489fc046ae86fd1dcb836a51b7` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:06:02.320486 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7128689a-8574-4c56-820d-0ce55d773b6e admin admin] RBAC: Authorizing `identity:delete_project(project_id=00e538489fc046ae86fd1dcb836a51b7)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:06:02.322716 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7128689a-8574-4c56-820d-0ce55d773b6e admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:06:02.374578 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-7128689a-8574-4c56-820d-0ce55d773b6e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '00e538489fc046ae86fd1dcb836a51b7', 'request_id': 'req-7128689a-8574-4c56-820d-0ce55d773b6e'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:06:02.383740 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-7128689a-8574-4c56-820d-0ce55d773b6e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '00e538489fc046ae86fd1dcb836a51b7', 'request_id': 'req-7128689a-8574-4c56-820d-0ce55d773b6e'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:06:02.390615 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-7128689a-8574-4c56-820d-0ce55d773b6e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '00e538489fc046ae86fd1dcb836a51b7', 'request_id': 'req-7128689a-8574-4c56-820d-0ce55d773b6e'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:06:02.397199 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:06:02.402512 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-7128689a-8574-4c56-820d-0ce55d773b6e admin admin] The token cache is being invalidate because project 00e538489fc046ae86fd1dcb836a51b7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81511) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 16:06:02.402910 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-7128689a-8574-4c56-820d-0ce55d773b6e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:06:02.404077 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-7128689a-8574-4c56-820d-0ce55d773b6e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:06:02.405976 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1346/2667] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Aug 30 16:06:02 2022] DELETE /identity/v3/projects/00e538489fc046ae86fd1dcb836a51b7 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:06:02.409962 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba632bd-baab-4455-8914-d29a70dfcdfe service neutron] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:06:02.419768 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba632bd-baab-4455-8914-d29a70dfcdfe service neutron] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:06:02.480186 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87d4d86-1ad1-4b80-b498-b80356f37ffd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:06:02.481200 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87d4d86-1ad1-4b80-b498-b80356f37ffd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:06:02.481517 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87d4d86-1ad1-4b80-b498-b80356f37ffd admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:06:02.481791 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87d4d86-1ad1-4b80-b498-b80356f37ffd admin admin] PATH_INFO: `/v3/users/8961468ecdd241c4920acbbb98401c11` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:06:02.483010 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87d4d86-1ad1-4b80-b498-b80356f37ffd admin admin] RBAC: Authorizing `identity:delete_user(user_id=8961468ecdd241c4920acbbb98401c11)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:06:02.485051 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87d4d86-1ad1-4b80-b498-b80356f37ffd admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:06:02.546957 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-c87d4d86-1ad1-4b80-b498-b80356f37ffd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8961468ecdd241c4920acbbb98401c11'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:06:02.556743 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-c87d4d86-1ad1-4b80-b498-b80356f37ffd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8961468ecdd241c4920acbbb98401c11'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:06:02.575256 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-c87d4d86-1ad1-4b80-b498-b80356f37ffd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8961468ecdd241c4920acbbb98401c11'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:06:02.582778 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-c87d4d86-1ad1-4b80-b498-b80356f37ffd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8961468ecdd241c4920acbbb98401c11'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:06:02.583577 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-c87d4d86-1ad1-4b80-b498-b80356f37ffd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8961468ecdd241c4920acbbb98401c11'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:06:02.589867 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1322/2668] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Aug 30 16:06:02 2022] DELETE /identity/v3/users/8961468ecdd241c4920acbbb98401c11 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:06:02.880099 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7128689a-8574-4c56-820d-0ce55d773b6e admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:06:02.881885 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-7128689a-8574-4c56-820d-0ce55d773b6e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:06:02.895611 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7128689a-8574-4c56-820d-0ce55d773b6e admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:06:02.957894 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3340e2c8-e841-45c0-8ec8-45bd8a9a78c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:06:02.958537 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3340e2c8-e841-45c0-8ec8-45bd8a9a78c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:06:02.958812 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3340e2c8-e841-45c0-8ec8-45bd8a9a78c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:06:02.959088 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3340e2c8-e841-45c0-8ec8-45bd8a9a78c8 admin admin] PATH_INFO: `/v3/projects/5570b36de34b4ff8b4612eda43aa8df1` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:06:02.960303 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3340e2c8-e841-45c0-8ec8-45bd8a9a78c8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5570b36de34b4ff8b4612eda43aa8df1)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:06:02.962380 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3340e2c8-e841-45c0-8ec8-45bd8a9a78c8 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:06:03.017099 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-3340e2c8-e841-45c0-8ec8-45bd8a9a78c8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5570b36de34b4ff8b4612eda43aa8df1', 'request_id': 'req-3340e2c8-e841-45c0-8ec8-45bd8a9a78c8'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:06:03.026062 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-3340e2c8-e841-45c0-8ec8-45bd8a9a78c8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5570b36de34b4ff8b4612eda43aa8df1', 'request_id': 'req-3340e2c8-e841-45c0-8ec8-45bd8a9a78c8'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:06:03.034355 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-3340e2c8-e841-45c0-8ec8-45bd8a9a78c8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5570b36de34b4ff8b4612eda43aa8df1', 'request_id': 'req-3340e2c8-e841-45c0-8ec8-45bd8a9a78c8'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:06:03.045547 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-3340e2c8-e841-45c0-8ec8-45bd8a9a78c8 admin admin] The token cache is being invalidate because project 5570b36de34b4ff8b4612eda43aa8df1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81511) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 16:06:03.045884 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-3340e2c8-e841-45c0-8ec8-45bd8a9a78c8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:06:03.046653 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-3340e2c8-e841-45c0-8ec8-45bd8a9a78c8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:06:03.048124 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1347/2669] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Aug 30 16:06:02 2022] DELETE /identity/v3/projects/5570b36de34b4ff8b4612eda43aa8df1 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:06:03.053359 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91271b44-2cc3-4d1a-9f8c-ee6ca0912dbc None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:06:03.053639 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91271b44-2cc3-4d1a-9f8c-ee6ca0912dbc None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:06:03.053915 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91271b44-2cc3-4d1a-9f8c-ee6ca0912dbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:06:03.202639 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-91271b44-2cc3-4d1a-9f8c-ee6ca0912dbc None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:06:03.203793 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-91271b44-2cc3-4d1a-9f8c-ee6ca0912dbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:06:03.243902 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:06:03.272037 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1323/2670] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 16:06:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 16:06:03.280085 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297c5575-9dd4-4f17-b5b7-eab8f3e3d3f0 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:06:03.280372 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297c5575-9dd4-4f17-b5b7-eab8f3e3d3f0 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:06:03.280647 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297c5575-9dd4-4f17-b5b7-eab8f3e3d3f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:06:03.306746 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-297c5575-9dd4-4f17-b5b7-eab8f3e3d3f0 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:06:03.308256 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-297c5575-9dd4-4f17-b5b7-eab8f3e3d3f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:06:03.318337 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1348/2671] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 16:06:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:06:03.325130 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91271b44-2cc3-4d1a-9f8c-ee6ca0912dbc None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:06:03.347486 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91271b44-2cc3-4d1a-9f8c-ee6ca0912dbc None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:06:03.358058 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cc5600-1194-4335-ac28-3a4f4312fcae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:06:03.358757 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cc5600-1194-4335-ac28-3a4f4312fcae admin admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:06:03.359032 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cc5600-1194-4335-ac28-3a4f4312fcae admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:06:03.359301 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cc5600-1194-4335-ac28-3a4f4312fcae admin admin] PATH_INFO: `/v3/domains` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:06:03.359945 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4cc5600-1194-4335-ac28-3a4f4312fcae admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81512) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:06:03.360244 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4cc5600-1194-4335-ac28-3a4f4312fcae admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:06:03.361898 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4cc5600-1194-4335-ac28-3a4f4312fcae admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:06:03.375065 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1324/2672] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Aug 30 16:06:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:06:03.381650 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297c5575-9dd4-4f17-b5b7-eab8f3e3d3f0 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:06:03.391439 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297c5575-9dd4-4f17-b5b7-eab8f3e3d3f0 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:06:03.396910 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d826862-71b7-4847-9654-6559d833d99c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:06:03.397664 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d826862-71b7-4847-9654-6559d833d99c admin admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:06:03.397942 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d826862-71b7-4847-9654-6559d833d99c admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:06:03.398215 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d826862-71b7-4847-9654-6559d833d99c admin admin] PATH_INFO: `/v3/projects` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:06:03.398864 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d826862-71b7-4847-9654-6559d833d99c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:06:03.400798 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d826862-71b7-4847-9654-6559d833d99c admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:06:03.431859 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1349/2673] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Aug 30 16:06:03 2022] POST /identity/v3/projects => generated 381 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 16:06:03.436111 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cc5600-1194-4335-ac28-3a4f4312fcae admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:06:03.446333 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cc5600-1194-4335-ac28-3a4f4312fcae admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:06:03.506405 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5b498e-9186-4c9e-999f-ae8ff6be0e33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:06:03.506948 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5b498e-9186-4c9e-999f-ae8ff6be0e33 admin admin] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:06:03.507227 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5b498e-9186-4c9e-999f-ae8ff6be0e33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:06:03.507511 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5b498e-9186-4c9e-999f-ae8ff6be0e33 admin admin] PATH_INFO: `/v3/users` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:06:03.508150 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5b498e-9186-4c9e-999f-ae8ff6be0e33 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:06:03.510175 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5b498e-9186-4c9e-999f-ae8ff6be0e33 admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:06:03.533964 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1325/2674] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Aug 30 16:06:03 2022] POST /identity/v3/users => generated 291 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 16:06:03.538466 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d826862-71b7-4847-9654-6559d833d99c admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:06:03.549448 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d826862-71b7-4847-9654-6559d833d99c admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:06:03.554397 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9822066e-a994-46f0-993c-74cf03893b0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:06:03.555008 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9822066e-a994-46f0-993c-74cf03893b0c admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:06:03.555283 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9822066e-a994-46f0-993c-74cf03893b0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:06:03.555553 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9822066e-a994-46f0-993c-74cf03893b0c admin admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:06:03.556199 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9822066e-a994-46f0-993c-74cf03893b0c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:06:03.557843 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9822066e-a994-46f0-993c-74cf03893b0c admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:06:03.567969 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1350/2675] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 30 16:06:03 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:06:03.572254 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5b498e-9186-4c9e-999f-ae8ff6be0e33 admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:06:03.582973 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5b498e-9186-4c9e-999f-ae8ff6be0e33 admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:06:03.587938 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4375b020-595f-47ad-a7f4-473819395453 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:06:03.588538 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4375b020-595f-47ad-a7f4-473819395453 admin admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:06:03.588818 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4375b020-595f-47ad-a7f4-473819395453 admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:06:03.589082 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4375b020-595f-47ad-a7f4-473819395453 admin admin] PATH_INFO: `/v3/roles` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:06:03.589805 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4375b020-595f-47ad-a7f4-473819395453 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:06:03.591411 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4375b020-595f-47ad-a7f4-473819395453 admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:06:03.601752 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1326/2676] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 30 16:06:03 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:06:03.606538 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9822066e-a994-46f0-993c-74cf03893b0c admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:06:03.616767 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9822066e-a994-46f0-993c-74cf03893b0c admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:06:03.621672 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cfff32-a869-4565-85c6-d8263526a146 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:06:03.622364 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cfff32-a869-4565-85c6-d8263526a146 admin admin] REQUEST_METHOD: `PUT` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:06:03.622661 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cfff32-a869-4565-85c6-d8263526a146 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:06:03.622926 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cfff32-a869-4565-85c6-d8263526a146 admin admin] PATH_INFO: `/v3/projects/b759a1f62e654920b0f6a5ea3b234fb6/users/1ce42783a8ec419488465113eab26c22/roles/df8f9f844b1b439181d75a07c1a93418` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:06:03.637973 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94cfff32-a869-4565-85c6-d8263526a146 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b759a1f62e654920b0f6a5ea3b234fb6, user_id=1ce42783a8ec419488465113eab26c22, role_id=df8f9f844b1b439181d75a07c1a93418)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:06:03.640684 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94cfff32-a869-4565-85c6-d8263526a146 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:06:03.656901 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1351/2677] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 30 16:06:03 2022] PUT /identity/v3/projects/b759a1f62e654920b0f6a5ea3b234fb6/users/1ce42783a8ec419488465113eab26c22/roles/df8f9f844b1b439181d75a07c1a93418 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:06:03.668253 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886ee2a5-eef8-46d3-b7bf-2a178bbed9ec None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:06:03.668538 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886ee2a5-eef8-46d3-b7bf-2a178bbed9ec None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:06:03.668810 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886ee2a5-eef8-46d3-b7bf-2a178bbed9ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:06:03.698419 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-886ee2a5-eef8-46d3-b7bf-2a178bbed9ec None None] MFA Rules not processed for user `1ce42783a8ec419488465113eab26c22`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:06:03.699647 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-886ee2a5-eef8-46d3-b7bf-2a178bbed9ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:06:03.767098 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1327/2678] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 16:06:03 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:06:03.772003 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cfff32-a869-4565-85c6-d8263526a146 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:06:03.799940 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:06:03.836458 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cfff32-a869-4565-85c6-d8263526a146 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:06:03.841444 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f54e056-0f54-4746-b611-3a7b3d5ee3ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:06:03.842056 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f54e056-0f54-4746-b611-3a7b3d5ee3ac admin admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:06:03.842331 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f54e056-0f54-4746-b611-3a7b3d5ee3ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:06:03.842641 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f54e056-0f54-4746-b611-3a7b3d5ee3ac admin admin] PATH_INFO: `/v3/projects` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:06:03.843276 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f54e056-0f54-4746-b611-3a7b3d5ee3ac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:06:03.845180 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f54e056-0f54-4746-b611-3a7b3d5ee3ac admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:06:03.875062 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1352/2679] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Aug 30 16:06:03 2022] POST /identity/v3/projects => generated 379 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:06:03.879402 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886ee2a5-eef8-46d3-b7bf-2a178bbed9ec None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:06:03.890417 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886ee2a5-eef8-46d3-b7bf-2a178bbed9ec None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:06:03.924274 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:06:03.953401 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76c71ef-3760-4e35-b18a-5abbdebb6e1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:06:03.954008 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76c71ef-3760-4e35-b18a-5abbdebb6e1f admin admin] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:06:03.954281 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76c71ef-3760-4e35-b18a-5abbdebb6e1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:06:03.954547 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76c71ef-3760-4e35-b18a-5abbdebb6e1f admin admin] PATH_INFO: `/v3/users` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:06:03.955296 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76c71ef-3760-4e35-b18a-5abbdebb6e1f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:06:03.957401 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76c71ef-3760-4e35-b18a-5abbdebb6e1f admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:06:03.983378 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1328/2680] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Aug 30 16:06:03 2022] POST /identity/v3/users => generated 296 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:06:03.987595 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f54e056-0f54-4746-b611-3a7b3d5ee3ac admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:06:03.999519 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f54e056-0f54-4746-b611-3a7b3d5ee3ac admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:06:04.005679 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e70fba-74e6-441c-8acd-401be4045428 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:06:04.006294 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e70fba-74e6-441c-8acd-401be4045428 admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:06:04.006570 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e70fba-74e6-441c-8acd-401be4045428 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:06:04.006830 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e70fba-74e6-441c-8acd-401be4045428 admin admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:06:04.007482 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e70fba-74e6-441c-8acd-401be4045428 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:06:04.009067 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e70fba-74e6-441c-8acd-401be4045428 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:06:04.019997 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1353/2681] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 30 16:06:03 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:06:04.025267 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76c71ef-3760-4e35-b18a-5abbdebb6e1f admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:06:04.042544 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76c71ef-3760-4e35-b18a-5abbdebb6e1f admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:06:04.048190 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bb39a9-bba4-415c-84b9-44177f64d633 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:06:04.048854 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bb39a9-bba4-415c-84b9-44177f64d633 admin admin] REQUEST_METHOD: `PUT` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:06:04.049148 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bb39a9-bba4-415c-84b9-44177f64d633 admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:06:04.049454 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bb39a9-bba4-415c-84b9-44177f64d633 admin admin] PATH_INFO: `/v3/projects/fc2e3c15e04243419b0e253e90c1fa7b/users/52c44673b4b648f3af7be100f25ec4e3/roles/f265386950a641ac8413b53eda4ae34d` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:06:04.067159 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25bb39a9-bba4-415c-84b9-44177f64d633 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fc2e3c15e04243419b0e253e90c1fa7b, user_id=52c44673b4b648f3af7be100f25ec4e3, role_id=f265386950a641ac8413b53eda4ae34d)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:06:04.069851 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25bb39a9-bba4-415c-84b9-44177f64d633 admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:06:04.084876 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1329/2682] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 30 16:06:04 2022] PUT /identity/v3/projects/fc2e3c15e04243419b0e253e90c1fa7b/users/52c44673b4b648f3af7be100f25ec4e3/roles/f265386950a641ac8413b53eda4ae34d => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 16:06:04.091093 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e70fba-74e6-441c-8acd-401be4045428 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:06:04.106780 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e70fba-74e6-441c-8acd-401be4045428 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:06:04.168233 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c26da4c-67a0-4128-bf40-f1175d632c9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:06:04.168851 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c26da4c-67a0-4128-bf40-f1175d632c9c admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:06:04.169117 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c26da4c-67a0-4128-bf40-f1175d632c9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:06:04.169410 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c26da4c-67a0-4128-bf40-f1175d632c9c admin admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:06:04.170052 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c26da4c-67a0-4128-bf40-f1175d632c9c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:06:04.171700 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c26da4c-67a0-4128-bf40-f1175d632c9c admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:06:04.182028 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1354/2683] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 30 16:06:04 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:06:04.186712 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bb39a9-bba4-415c-84b9-44177f64d633 admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:06:04.197086 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bb39a9-bba4-415c-84b9-44177f64d633 admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:06:04.202198 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a40813-546e-44d0-acd1-548b63b9ced2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:06:04.202866 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a40813-546e-44d0-acd1-548b63b9ced2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:06:04.203157 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a40813-546e-44d0-acd1-548b63b9ced2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:06:04.203418 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a40813-546e-44d0-acd1-548b63b9ced2 admin admin] PATH_INFO: `/v3/projects/fc2e3c15e04243419b0e253e90c1fa7b/users/52c44673b4b648f3af7be100f25ec4e3/roles/f265386950a641ac8413b53eda4ae34d` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:06:04.205339 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a40813-546e-44d0-acd1-548b63b9ced2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fc2e3c15e04243419b0e253e90c1fa7b, user_id=52c44673b4b648f3af7be100f25ec4e3, role_id=f265386950a641ac8413b53eda4ae34d)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:06:04.208178 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a40813-546e-44d0-acd1-548b63b9ced2 admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:06:04.219042 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1330/2684] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 30 16:06:04 2022] PUT /identity/v3/projects/fc2e3c15e04243419b0e253e90c1fa7b/users/52c44673b4b648f3af7be100f25ec4e3/roles/f265386950a641ac8413b53eda4ae34d => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:06:04.230276 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395b8f6a-02c8-4136-9f2a-e5efae1b61da None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:06:04.230562 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395b8f6a-02c8-4136-9f2a-e5efae1b61da None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:06:04.231180 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395b8f6a-02c8-4136-9f2a-e5efae1b61da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:06:04.260451 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-395b8f6a-02c8-4136-9f2a-e5efae1b61da None None] MFA Rules not processed for user `52c44673b4b648f3af7be100f25ec4e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:06:04.261738 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-395b8f6a-02c8-4136-9f2a-e5efae1b61da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:06:04.356462 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1355/2685] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 16:06:04 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:06:04.369833 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a40813-546e-44d0-acd1-548b63b9ced2 admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:06:04.372911 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-33a40813-546e-44d0-acd1-548b63b9ced2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:06:04.443201 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a40813-546e-44d0-acd1-548b63b9ced2 admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:06:04.447070 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846dd26a-30ce-41cd-8332-66fcbb258850 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0100a04821804993866ea20f159663f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:06:04.447678 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846dd26a-30ce-41cd-8332-66fcbb258850 service cinder] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:06:04.447953 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846dd26a-30ce-41cd-8332-66fcbb258850 service cinder] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:06:04.448222 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846dd26a-30ce-41cd-8332-66fcbb258850 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:06:04.457504 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846dd26a-30ce-41cd-8332-66fcbb258850 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:06:04.459063 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846dd26a-30ce-41cd-8332-66fcbb258850 service cinder] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:06:04.469166 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1331/2686] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 16:06:04 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:06:04.557716 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395b8f6a-02c8-4136-9f2a-e5efae1b61da None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:06:04.568688 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395b8f6a-02c8-4136-9f2a-e5efae1b61da None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:06:04.572539 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc540374-a7e6-4f0b-ab63-81e7ad678138 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0100a04821804993866ea20f159663f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:06:04.573145 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc540374-a7e6-4f0b-ab63-81e7ad678138 service cinder] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:06:04.573450 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc540374-a7e6-4f0b-ab63-81e7ad678138 service cinder] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:06:04.573714 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc540374-a7e6-4f0b-ab63-81e7ad678138 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:06:04.592015 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:06:04.635614 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc540374-a7e6-4f0b-ab63-81e7ad678138 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:06:04.637081 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc540374-a7e6-4f0b-ab63-81e7ad678138 service cinder] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:06:04.648497 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1356/2687] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 16:06:04 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:06:14.464590 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=54,keystone:INSERT=5 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:06:14.642083 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=19 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:06:14.681482 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846dd26a-30ce-41cd-8332-66fcbb258850 service cinder] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:06:14.747199 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846dd26a-30ce-41cd-8332-66fcbb258850 service cinder] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:06:14.755640 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6e3fec-369c-480d-b62c-0b626fef8ff8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:06:14.756552 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6e3fec-369c-480d-b62c-0b626fef8ff8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:06:14.756831 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6e3fec-369c-480d-b62c-0b626fef8ff8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:06:14.757100 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6e3fec-369c-480d-b62c-0b626fef8ff8 admin admin] PATH_INFO: `/v3/users/1ce42783a8ec419488465113eab26c22` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:06:14.758419 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6e3fec-369c-480d-b62c-0b626fef8ff8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1ce42783a8ec419488465113eab26c22)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:06:14.760363 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6e3fec-369c-480d-b62c-0b626fef8ff8 admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:06:14.826387 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-bc6e3fec-369c-480d-b62c-0b626fef8ff8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1ce42783a8ec419488465113eab26c22'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:06:14.835995 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-bc6e3fec-369c-480d-b62c-0b626fef8ff8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1ce42783a8ec419488465113eab26c22'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:06:14.847736 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-bc6e3fec-369c-480d-b62c-0b626fef8ff8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1ce42783a8ec419488465113eab26c22'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:06:14.854585 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-bc6e3fec-369c-480d-b62c-0b626fef8ff8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1ce42783a8ec419488465113eab26c22'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:06:14.855224 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-bc6e3fec-369c-480d-b62c-0b626fef8ff8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1ce42783a8ec419488465113eab26c22'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:06:14.861461 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1332/2688] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Aug 30 16:06:14 2022] DELETE /identity/v3/users/1ce42783a8ec419488465113eab26c22 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:06:14.866022 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a27f18b-4cf3-47ca-9538-c7dd847c05a6 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:06:14.866302 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a27f18b-4cf3-47ca-9538-c7dd847c05a6 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:06:14.866570 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a27f18b-4cf3-47ca-9538-c7dd847c05a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:06:14.886687 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-5a27f18b-4cf3-47ca-9538-c7dd847c05a6 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:06:14.887924 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-5a27f18b-4cf3-47ca-9538-c7dd847c05a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:06:14.951049 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1357/2689] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 16:06:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:06:14.961051 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6e3fec-369c-480d-b62c-0b626fef8ff8 admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:06:14.962565 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-bc6e3fec-369c-480d-b62c-0b626fef8ff8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:06:15.023419 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6e3fec-369c-480d-b62c-0b626fef8ff8 admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:06:15.027158 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3c28df-2ec4-4e55-9dd5-0284c40ad804 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f4157105f3488f807b7475008ab722', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:06:15.027764 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3c28df-2ec4-4e55-9dd5-0284c40ad804 service neutron] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:06:15.028036 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3c28df-2ec4-4e55-9dd5-0284c40ad804 service neutron] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:06:15.028306 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3c28df-2ec4-4e55-9dd5-0284c40ad804 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:06:15.038381 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3c28df-2ec4-4e55-9dd5-0284c40ad804 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:06:15.039886 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3c28df-2ec4-4e55-9dd5-0284c40ad804 service neutron] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:06:15.050039 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1333/2690] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 16:06:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:06:15.327400 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a27f18b-4cf3-47ca-9538-c7dd847c05a6 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:06:15.328978 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-5a27f18b-4cf3-47ca-9538-c7dd847c05a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:06:15.342005 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a27f18b-4cf3-47ca-9538-c7dd847c05a6 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:06:15.347924 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b5cc9d-e2f8-49c0-a2e9-db1bb0665292 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:06:15.348663 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b5cc9d-e2f8-49c0-a2e9-db1bb0665292 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:06:15.348992 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b5cc9d-e2f8-49c0-a2e9-db1bb0665292 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:06:15.349354 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b5cc9d-e2f8-49c0-a2e9-db1bb0665292 admin admin] PATH_INFO: `/v3/projects/b759a1f62e654920b0f6a5ea3b234fb6` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:06:15.350672 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b5cc9d-e2f8-49c0-a2e9-db1bb0665292 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b759a1f62e654920b0f6a5ea3b234fb6)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:06:15.352596 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b5cc9d-e2f8-49c0-a2e9-db1bb0665292 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:06:15.403255 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-e7b5cc9d-e2f8-49c0-a2e9-db1bb0665292 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b759a1f62e654920b0f6a5ea3b234fb6', 'request_id': 'req-e7b5cc9d-e2f8-49c0-a2e9-db1bb0665292'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:06:15.412160 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-e7b5cc9d-e2f8-49c0-a2e9-db1bb0665292 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b759a1f62e654920b0f6a5ea3b234fb6', 'request_id': 'req-e7b5cc9d-e2f8-49c0-a2e9-db1bb0665292'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:06:15.421617 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-e7b5cc9d-e2f8-49c0-a2e9-db1bb0665292 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b759a1f62e654920b0f6a5ea3b234fb6', 'request_id': 'req-e7b5cc9d-e2f8-49c0-a2e9-db1bb0665292'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:06:15.433440 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-e7b5cc9d-e2f8-49c0-a2e9-db1bb0665292 admin admin] The token cache is being invalidate because project b759a1f62e654920b0f6a5ea3b234fb6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81511) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 16:06:15.434466 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-e7b5cc9d-e2f8-49c0-a2e9-db1bb0665292 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:06:15.434466 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-e7b5cc9d-e2f8-49c0-a2e9-db1bb0665292 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:06:15.436010 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1358/2691] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Aug 30 16:06:15 2022] DELETE /identity/v3/projects/b759a1f62e654920b0f6a5ea3b234fb6 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 16:06:15.440195 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3c28df-2ec4-4e55-9dd5-0284c40ad804 service neutron] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:06:15.451030 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3c28df-2ec4-4e55-9dd5-0284c40ad804 service neutron] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:06:15.470646 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:06:15.513036 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70d049d-0c1f-479c-8946-1c2d5ded1364 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:06:15.513695 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70d049d-0c1f-479c-8946-1c2d5ded1364 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:06:15.513952 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70d049d-0c1f-479c-8946-1c2d5ded1364 admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:06:15.514218 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70d049d-0c1f-479c-8946-1c2d5ded1364 admin admin] PATH_INFO: `/v3/users/52c44673b4b648f3af7be100f25ec4e3` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:06:15.515382 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70d049d-0c1f-479c-8946-1c2d5ded1364 admin admin] RBAC: Authorizing `identity:delete_user(user_id=52c44673b4b648f3af7be100f25ec4e3)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:06:15.517348 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70d049d-0c1f-479c-8946-1c2d5ded1364 admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:06:15.567101 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-a70d049d-0c1f-479c-8946-1c2d5ded1364 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '52c44673b4b648f3af7be100f25ec4e3'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:06:15.576656 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-a70d049d-0c1f-479c-8946-1c2d5ded1364 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '52c44673b4b648f3af7be100f25ec4e3'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:06:15.588156 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-a70d049d-0c1f-479c-8946-1c2d5ded1364 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '52c44673b4b648f3af7be100f25ec4e3'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:06:15.594893 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-a70d049d-0c1f-479c-8946-1c2d5ded1364 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '52c44673b4b648f3af7be100f25ec4e3'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:06:15.595558 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-a70d049d-0c1f-479c-8946-1c2d5ded1364 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '52c44673b4b648f3af7be100f25ec4e3'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:06:15.601549 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1334/2692] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Aug 30 16:06:15 2022] DELETE /identity/v3/users/52c44673b4b648f3af7be100f25ec4e3 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:06:16.074930 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b5cc9d-e2f8-49c0-a2e9-db1bb0665292 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:06:16.077938 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-e7b5cc9d-e2f8-49c0-a2e9-db1bb0665292 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:06:16.096237 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b5cc9d-e2f8-49c0-a2e9-db1bb0665292 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:06:16.210517 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1234aa-e2fc-4644-aae1-0fd365a118c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:06:16.211150 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1234aa-e2fc-4644-aae1-0fd365a118c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:06:16.212067 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1234aa-e2fc-4644-aae1-0fd365a118c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:06:16.212067 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1234aa-e2fc-4644-aae1-0fd365a118c6 admin admin] PATH_INFO: `/v3/projects/fc2e3c15e04243419b0e253e90c1fa7b` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:06:16.213015 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1234aa-e2fc-4644-aae1-0fd365a118c6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fc2e3c15e04243419b0e253e90c1fa7b)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:06:16.215029 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1234aa-e2fc-4644-aae1-0fd365a118c6 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:06:16.254389 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:06:16.283658 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-cd1234aa-e2fc-4644-aae1-0fd365a118c6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fc2e3c15e04243419b0e253e90c1fa7b', 'request_id': 'req-cd1234aa-e2fc-4644-aae1-0fd365a118c6'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:06:16.292516 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-cd1234aa-e2fc-4644-aae1-0fd365a118c6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fc2e3c15e04243419b0e253e90c1fa7b', 'request_id': 'req-cd1234aa-e2fc-4644-aae1-0fd365a118c6'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:06:16.299871 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-cd1234aa-e2fc-4644-aae1-0fd365a118c6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fc2e3c15e04243419b0e253e90c1fa7b', 'request_id': 'req-cd1234aa-e2fc-4644-aae1-0fd365a118c6'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:06:16.310452 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-cd1234aa-e2fc-4644-aae1-0fd365a118c6 admin admin] The token cache is being invalidate because project fc2e3c15e04243419b0e253e90c1fa7b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81511) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 16:06:16.310740 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-cd1234aa-e2fc-4644-aae1-0fd365a118c6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:06:16.311418 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-cd1234aa-e2fc-4644-aae1-0fd365a118c6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:06:16.312935 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1359/2693] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Aug 30 16:06:16 2022] DELETE /identity/v3/projects/fc2e3c15e04243419b0e253e90c1fa7b => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:06:16.317894 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c6b484-3d8d-4110-98f7-5ee14e83e0c7 None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:06:16.318180 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c6b484-3d8d-4110-98f7-5ee14e83e0c7 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:06:16.318512 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c6b484-3d8d-4110-98f7-5ee14e83e0c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:06:16.337879 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-89c6b484-3d8d-4110-98f7-5ee14e83e0c7 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:06:16.339092 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-89c6b484-3d8d-4110-98f7-5ee14e83e0c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:06:16.401921 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1335/2694] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 16:06:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:06:16.410935 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669fb878-734d-44bb-99dc-6d32817837c5 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:06:16.411201 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669fb878-734d-44bb-99dc-6d32817837c5 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:06:16.411465 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669fb878-734d-44bb-99dc-6d32817837c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:06:16.431801 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-669fb878-734d-44bb-99dc-6d32817837c5 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:06:16.433028 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-669fb878-734d-44bb-99dc-6d32817837c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:06:16.444234 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1360/2695] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 16:06:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:06:16.445452 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c6b484-3d8d-4110-98f7-5ee14e83e0c7 None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:06:16.455510 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c6b484-3d8d-4110-98f7-5ee14e83e0c7 None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:06:16.463356 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d640339e-44f0-4d31-ab6e-4955a651de37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:06:16.463988 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d640339e-44f0-4d31-ab6e-4955a651de37 admin admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:06:16.464265 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d640339e-44f0-4d31-ab6e-4955a651de37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:06:16.464532 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d640339e-44f0-4d31-ab6e-4955a651de37 admin admin] PATH_INFO: `/v3/domains` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:06:16.465195 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d640339e-44f0-4d31-ab6e-4955a651de37 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81512) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:06:16.465533 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d640339e-44f0-4d31-ab6e-4955a651de37 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:06:16.467162 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d640339e-44f0-4d31-ab6e-4955a651de37 admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:06:16.479888 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1336/2696] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Aug 30 16:06:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 16:06:16.484619 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669fb878-734d-44bb-99dc-6d32817837c5 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:06:16.494112 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669fb878-734d-44bb-99dc-6d32817837c5 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:06:16.498671 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e2ec7b-067d-42a7-bbdc-2f9cb8478f30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:06:16.499273 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e2ec7b-067d-42a7-bbdc-2f9cb8478f30 admin admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:06:16.499546 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e2ec7b-067d-42a7-bbdc-2f9cb8478f30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:06:16.499811 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e2ec7b-067d-42a7-bbdc-2f9cb8478f30 admin admin] PATH_INFO: `/v3/projects` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:06:16.500456 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e2ec7b-067d-42a7-bbdc-2f9cb8478f30 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:06:16.502387 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e2ec7b-067d-42a7-bbdc-2f9cb8478f30 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:06:16.533291 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1361/2697] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Aug 30 16:06:16 2022] POST /identity/v3/projects => generated 387 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:06:16.534849 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d640339e-44f0-4d31-ab6e-4955a651de37 admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:06:16.545163 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d640339e-44f0-4d31-ab6e-4955a651de37 admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:06:16.572837 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:06:16.607377 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923c2293-6e36-48e6-b9b1-8c20027f9094 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:06:16.607983 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923c2293-6e36-48e6-b9b1-8c20027f9094 admin admin] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:06:16.608257 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923c2293-6e36-48e6-b9b1-8c20027f9094 admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:06:16.608527 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923c2293-6e36-48e6-b9b1-8c20027f9094 admin admin] PATH_INFO: `/v3/users` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:06:16.609162 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923c2293-6e36-48e6-b9b1-8c20027f9094 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:06:16.611136 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923c2293-6e36-48e6-b9b1-8c20027f9094 admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:06:16.635321 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1337/2698] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Aug 30 16:06:16 2022] POST /identity/v3/users => generated 294 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:06:16.640447 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e2ec7b-067d-42a7-bbdc-2f9cb8478f30 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:06:16.652202 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e2ec7b-067d-42a7-bbdc-2f9cb8478f30 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:06:16.656883 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fe98e7-7e01-4853-ac16-b86d0a012236 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:06:16.657519 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fe98e7-7e01-4853-ac16-b86d0a012236 admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:06:16.657788 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fe98e7-7e01-4853-ac16-b86d0a012236 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:06:16.658050 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fe98e7-7e01-4853-ac16-b86d0a012236 admin admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:06:16.658690 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fe98e7-7e01-4853-ac16-b86d0a012236 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:06:16.660585 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fe98e7-7e01-4853-ac16-b86d0a012236 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:06:16.670956 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1362/2699] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 30 16:06:16 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:06:16.675053 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923c2293-6e36-48e6-b9b1-8c20027f9094 admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:06:16.686630 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923c2293-6e36-48e6-b9b1-8c20027f9094 admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:06:16.691828 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08ffc6e-d27a-4f76-ad97-77ac511f6b4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:06:16.692489 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08ffc6e-d27a-4f76-ad97-77ac511f6b4e admin admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:06:16.692908 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08ffc6e-d27a-4f76-ad97-77ac511f6b4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:06:16.693680 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08ffc6e-d27a-4f76-ad97-77ac511f6b4e admin admin] PATH_INFO: `/v3/roles` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:06:16.693897 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08ffc6e-d27a-4f76-ad97-77ac511f6b4e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:06:16.695496 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08ffc6e-d27a-4f76-ad97-77ac511f6b4e admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:06:16.713506 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1338/2700] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 30 16:06:16 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:06:16.715430 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fe98e7-7e01-4853-ac16-b86d0a012236 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:06:16.725999 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fe98e7-7e01-4853-ac16-b86d0a012236 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:06:16.731333 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd02c14-5c0a-42b0-a273-47f0aa841ebf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:06:16.732112 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd02c14-5c0a-42b0-a273-47f0aa841ebf admin admin] REQUEST_METHOD: `PUT` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:06:16.732481 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd02c14-5c0a-42b0-a273-47f0aa841ebf admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:06:16.732891 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd02c14-5c0a-42b0-a273-47f0aa841ebf admin admin] PATH_INFO: `/v3/projects/c9c78ee2aa804862ad75a40bcf262124/users/d76bd90a6ca2470c958ee196d2d50acf/roles/df8f9f844b1b439181d75a07c1a93418` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:06:16.748711 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08ffc6e-d27a-4f76-ad97-77ac511f6b4e admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:06:16.750980 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-d08ffc6e-d27a-4f76-ad97-77ac511f6b4e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:06:16.758601 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd02c14-5c0a-42b0-a273-47f0aa841ebf admin admin] RBAC: Authorizing `identity:create_grant(project_id=c9c78ee2aa804862ad75a40bcf262124, user_id=d76bd90a6ca2470c958ee196d2d50acf, role_id=df8f9f844b1b439181d75a07c1a93418)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:06:16.761905 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd02c14-5c0a-42b0-a273-47f0aa841ebf admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:06:16.794845 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1363/2701] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 30 16:06:16 2022] PUT /identity/v3/projects/c9c78ee2aa804862ad75a40bcf262124/users/d76bd90a6ca2470c958ee196d2d50acf/roles/df8f9f844b1b439181d75a07c1a93418 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:06:16.796781 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63eadbf-2973-4b5c-9a20-4262e95aed7c None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:06:16.797135 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63eadbf-2973-4b5c-9a20-4262e95aed7c None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:06:16.797570 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63eadbf-2973-4b5c-9a20-4262e95aed7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:06:16.838283 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-a63eadbf-2973-4b5c-9a20-4262e95aed7c None None] MFA Rules not processed for user `d76bd90a6ca2470c958ee196d2d50acf`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:06:16.842374 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-a63eadbf-2973-4b5c-9a20-4262e95aed7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:06:16.858741 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08ffc6e-d27a-4f76-ad97-77ac511f6b4e admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:06:16.961007 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1364/2702] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 16:06:16 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:06:16.965698 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63eadbf-2973-4b5c-9a20-4262e95aed7c None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:06:16.966741 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ff8999-4c93-4e9e-ab69-a9570366bede service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a1e756ae9be4a7bac1e81acf07e8f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:06:16.967551 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ff8999-4c93-4e9e-ab69-a9570366bede service nova] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:06:16.967889 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ff8999-4c93-4e9e-ab69-a9570366bede service nova] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:06:16.968224 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ff8999-4c93-4e9e-ab69-a9570366bede service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:06:16.970439 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-b8ff8999-4c93-4e9e-ab69-a9570366bede service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:06:16.984382 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ff8999-4c93-4e9e-ab69-a9570366bede service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:06:16.986446 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ff8999-4c93-4e9e-ab69-a9570366bede service nova] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:06:16.999957 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1339/2703] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 16:06:16 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 252 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:06:17.030112 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:DELETE=6,keystone:SELECT=91,keystone:INSERT=3 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:06:17.063036 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63eadbf-2973-4b5c-9a20-4262e95aed7c None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:06:17.068685 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a362fb-9b00-41bd-9be3-95086210c5b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:06:17.069815 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a362fb-9b00-41bd-9be3-95086210c5b3 admin admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:06:17.070181 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a362fb-9b00-41bd-9be3-95086210c5b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:06:17.070547 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a362fb-9b00-41bd-9be3-95086210c5b3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:06:17.072249 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a362fb-9b00-41bd-9be3-95086210c5b3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:06:17.074582 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a362fb-9b00-41bd-9be3-95086210c5b3 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:06:17.109515 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1365/2704] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Aug 30 16:06:16 2022] POST /identity/v3/projects => generated 385 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:06:17.114439 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ff8999-4c93-4e9e-ab69-a9570366bede service nova] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:06:17.127544 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ff8999-4c93-4e9e-ab69-a9570366bede service nova] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:06:17.160154 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:06:17.189035 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a57e7b-adb2-43be-8861-cb163d08e042 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:06:17.189680 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a57e7b-adb2-43be-8861-cb163d08e042 admin admin] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:06:17.189962 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a57e7b-adb2-43be-8861-cb163d08e042 admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:06:17.190235 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a57e7b-adb2-43be-8861-cb163d08e042 admin admin] PATH_INFO: `/v3/users` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:06:17.190862 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a57e7b-adb2-43be-8861-cb163d08e042 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:06:17.192794 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a57e7b-adb2-43be-8861-cb163d08e042 admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:06:17.217760 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1340/2705] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Aug 30 16:06:17 2022] POST /identity/v3/users => generated 299 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:06:17.221752 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a362fb-9b00-41bd-9be3-95086210c5b3 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:06:17.233752 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a362fb-9b00-41bd-9be3-95086210c5b3 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:06:17.238424 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d06573-0cb4-4b8a-b2a1-3ae397388894 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:06:17.239059 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d06573-0cb4-4b8a-b2a1-3ae397388894 admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:06:17.239333 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d06573-0cb4-4b8a-b2a1-3ae397388894 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:06:17.239597 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d06573-0cb4-4b8a-b2a1-3ae397388894 admin admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:06:17.240274 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d06573-0cb4-4b8a-b2a1-3ae397388894 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:06:17.241930 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d06573-0cb4-4b8a-b2a1-3ae397388894 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:06:17.252137 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1366/2706] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 30 16:06:17 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:06:17.256564 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a57e7b-adb2-43be-8861-cb163d08e042 admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:06:17.266936 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a57e7b-adb2-43be-8861-cb163d08e042 admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:06:17.272463 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73587e56-f517-4f28-8f5c-742f281479a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:06:17.273255 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73587e56-f517-4f28-8f5c-742f281479a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:06:17.273551 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73587e56-f517-4f28-8f5c-742f281479a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:06:17.273844 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73587e56-f517-4f28-8f5c-742f281479a6 admin admin] PATH_INFO: `/v3/projects/9806a12f33c64782bf5038d3e4908440/users/a7093cbd398e4ba7ae8a4bb2604f022c/roles/f265386950a641ac8413b53eda4ae34d` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:06:17.289499 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73587e56-f517-4f28-8f5c-742f281479a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9806a12f33c64782bf5038d3e4908440, user_id=a7093cbd398e4ba7ae8a4bb2604f022c, role_id=f265386950a641ac8413b53eda4ae34d)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:06:17.292163 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73587e56-f517-4f28-8f5c-742f281479a6 admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:06:17.308560 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1341/2707] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 30 16:06:17 2022] PUT /identity/v3/projects/9806a12f33c64782bf5038d3e4908440/users/a7093cbd398e4ba7ae8a4bb2604f022c/roles/f265386950a641ac8413b53eda4ae34d => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 16:06:17.312762 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d06573-0cb4-4b8a-b2a1-3ae397388894 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:06:17.335981 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d06573-0cb4-4b8a-b2a1-3ae397388894 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:06:17.409694 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2b31e3-2423-417c-9b13-c90a09ec52c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:06:17.410432 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2b31e3-2423-417c-9b13-c90a09ec52c0 admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:06:17.410714 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2b31e3-2423-417c-9b13-c90a09ec52c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:06:17.410989 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2b31e3-2423-417c-9b13-c90a09ec52c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:06:17.411637 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2b31e3-2423-417c-9b13-c90a09ec52c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:06:17.413380 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2b31e3-2423-417c-9b13-c90a09ec52c0 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:06:17.424459 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1367/2708] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 30 16:06:17 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:06:17.428597 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73587e56-f517-4f28-8f5c-742f281479a6 admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:06:17.449103 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73587e56-f517-4f28-8f5c-742f281479a6 admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:06:17.454789 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e8183d-1159-48d8-8c5c-428fd26b0ae2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:06:17.455451 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e8183d-1159-48d8-8c5c-428fd26b0ae2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:06:17.455733 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e8183d-1159-48d8-8c5c-428fd26b0ae2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:06:17.456006 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e8183d-1159-48d8-8c5c-428fd26b0ae2 admin admin] PATH_INFO: `/v3/projects/9806a12f33c64782bf5038d3e4908440/users/a7093cbd398e4ba7ae8a4bb2604f022c/roles/f265386950a641ac8413b53eda4ae34d` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:06:17.457986 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e8183d-1159-48d8-8c5c-428fd26b0ae2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9806a12f33c64782bf5038d3e4908440, user_id=a7093cbd398e4ba7ae8a4bb2604f022c, role_id=f265386950a641ac8413b53eda4ae34d)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:06:17.460733 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e8183d-1159-48d8-8c5c-428fd26b0ae2 admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:06:17.471601 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1342/2709] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 30 16:06:17 2022] PUT /identity/v3/projects/9806a12f33c64782bf5038d3e4908440/users/a7093cbd398e4ba7ae8a4bb2604f022c/roles/f265386950a641ac8413b53eda4ae34d => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:06:17.485641 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce156ac-6a28-4467-9082-7f22f9d70aa4 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:06:17.486165 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce156ac-6a28-4467-9082-7f22f9d70aa4 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:06:17.486547 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce156ac-6a28-4467-9082-7f22f9d70aa4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:06:17.519631 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-5ce156ac-6a28-4467-9082-7f22f9d70aa4 None None] MFA Rules not processed for user `a7093cbd398e4ba7ae8a4bb2604f022c`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:06:17.520617 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-5ce156ac-6a28-4467-9082-7f22f9d70aa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:06:17.610692 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1368/2710] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 16:06:17 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:06:17.625093 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e8183d-1159-48d8-8c5c-428fd26b0ae2 admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:06:17.630861 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-a8e8183d-1159-48d8-8c5c-428fd26b0ae2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:06:17.696543 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e8183d-1159-48d8-8c5c-428fd26b0ae2 admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:06:17.700442 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0191240-c163-4fb5-b1de-c54b7d0bd248 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0100a04821804993866ea20f159663f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:06:17.701048 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0191240-c163-4fb5-b1de-c54b7d0bd248 service cinder] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:06:17.701374 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0191240-c163-4fb5-b1de-c54b7d0bd248 service cinder] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:06:17.701634 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0191240-c163-4fb5-b1de-c54b7d0bd248 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:06:17.711848 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0191240-c163-4fb5-b1de-c54b7d0bd248 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:06:17.713358 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0191240-c163-4fb5-b1de-c54b7d0bd248 service cinder] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:06:17.723729 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1343/2711] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 16:06:17 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:06:17.754257 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce156ac-6a28-4467-9082-7f22f9d70aa4 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:06:17.798515 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:06:17.825574 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce156ac-6a28-4467-9082-7f22f9d70aa4 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:06:17.830116 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d8ecec-c66e-401d-b0b7-4307de8b7ab4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:06:17.830766 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d8ecec-c66e-401d-b0b7-4307de8b7ab4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:06:17.831042 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d8ecec-c66e-401d-b0b7-4307de8b7ab4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:06:17.831317 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d8ecec-c66e-401d-b0b7-4307de8b7ab4 admin admin] PATH_INFO: `/v3/users/d76bd90a6ca2470c958ee196d2d50acf` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:06:17.832469 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d8ecec-c66e-401d-b0b7-4307de8b7ab4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d76bd90a6ca2470c958ee196d2d50acf)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:06:17.834488 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d8ecec-c66e-401d-b0b7-4307de8b7ab4 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:06:17.889811 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-e4d8ecec-c66e-401d-b0b7-4307de8b7ab4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd76bd90a6ca2470c958ee196d2d50acf'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:06:17.905498 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-e4d8ecec-c66e-401d-b0b7-4307de8b7ab4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd76bd90a6ca2470c958ee196d2d50acf'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:06:17.914917 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-e4d8ecec-c66e-401d-b0b7-4307de8b7ab4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd76bd90a6ca2470c958ee196d2d50acf'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:06:17.915655 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-e4d8ecec-c66e-401d-b0b7-4307de8b7ab4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd76bd90a6ca2470c958ee196d2d50acf'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:06:17.923674 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-e4d8ecec-c66e-401d-b0b7-4307de8b7ab4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd76bd90a6ca2470c958ee196d2d50acf'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:06:17.929721 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1369/2712] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Aug 30 16:06:17 2022] DELETE /identity/v3/users/d76bd90a6ca2470c958ee196d2d50acf => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:06:17.934543 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c141dd-43a8-4ab9-a82b-fefe9b613609 None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:06:17.934829 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c141dd-43a8-4ab9-a82b-fefe9b613609 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:06:17.935102 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c141dd-43a8-4ab9-a82b-fefe9b613609 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:06:17.954776 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-c1c141dd-43a8-4ab9-a82b-fefe9b613609 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:06:17.956104 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-c1c141dd-43a8-4ab9-a82b-fefe9b613609 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:06:18.017530 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1344/2713] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 16:06:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 16:06:18.026522 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d8ecec-c66e-401d-b0b7-4307de8b7ab4 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:06:18.028139 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-e4d8ecec-c66e-401d-b0b7-4307de8b7ab4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:06:18.092694 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d8ecec-c66e-401d-b0b7-4307de8b7ab4 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:06:18.096485 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d779f8fc-2b35-469b-b4b0-78d309667b92 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f4157105f3488f807b7475008ab722', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:06:18.097250 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d779f8fc-2b35-469b-b4b0-78d309667b92 service neutron] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:06:18.097610 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d779f8fc-2b35-469b-b4b0-78d309667b92 service neutron] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:06:18.097994 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d779f8fc-2b35-469b-b4b0-78d309667b92 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:06:18.110055 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d779f8fc-2b35-469b-b4b0-78d309667b92 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:06:18.111584 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d779f8fc-2b35-469b-b4b0-78d309667b92 service neutron] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:06:18.122010 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1370/2714] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 16:06:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:06:18.402355 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c141dd-43a8-4ab9-a82b-fefe9b613609 None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:06:18.404016 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-c1c141dd-43a8-4ab9-a82b-fefe9b613609 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:06:18.417546 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c141dd-43a8-4ab9-a82b-fefe9b613609 None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:06:18.422336 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4f78f1-45b3-4a4d-9513-3c428fff8dd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:06:18.422999 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4f78f1-45b3-4a4d-9513-3c428fff8dd5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:06:18.423275 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4f78f1-45b3-4a4d-9513-3c428fff8dd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:06:18.423549 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4f78f1-45b3-4a4d-9513-3c428fff8dd5 admin admin] PATH_INFO: `/v3/projects/c9c78ee2aa804862ad75a40bcf262124` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:06:18.424829 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4f78f1-45b3-4a4d-9513-3c428fff8dd5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c9c78ee2aa804862ad75a40bcf262124)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:06:18.426810 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4f78f1-45b3-4a4d-9513-3c428fff8dd5 admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:06:18.453786 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:06:18.481273 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-2e4f78f1-45b3-4a4d-9513-3c428fff8dd5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c9c78ee2aa804862ad75a40bcf262124', 'request_id': 'req-2e4f78f1-45b3-4a4d-9513-3c428fff8dd5'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:06:18.490068 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-2e4f78f1-45b3-4a4d-9513-3c428fff8dd5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c9c78ee2aa804862ad75a40bcf262124', 'request_id': 'req-2e4f78f1-45b3-4a4d-9513-3c428fff8dd5'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:06:18.497010 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-2e4f78f1-45b3-4a4d-9513-3c428fff8dd5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c9c78ee2aa804862ad75a40bcf262124', 'request_id': 'req-2e4f78f1-45b3-4a4d-9513-3c428fff8dd5'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:06:18.509260 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-2e4f78f1-45b3-4a4d-9513-3c428fff8dd5 admin admin] The token cache is being invalidate because project c9c78ee2aa804862ad75a40bcf262124 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81512) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 16:06:18.509676 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-2e4f78f1-45b3-4a4d-9513-3c428fff8dd5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:06:18.510512 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-2e4f78f1-45b3-4a4d-9513-3c428fff8dd5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:06:18.512137 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1345/2715] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Aug 30 16:06:18 2022] DELETE /identity/v3/projects/c9c78ee2aa804862ad75a40bcf262124 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:06:18.516253 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d779f8fc-2b35-469b-b4b0-78d309667b92 service neutron] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:06:18.526507 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d779f8fc-2b35-469b-b4b0-78d309667b92 service neutron] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:06:18.576557 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:06:18.583086 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848c3f6b-01dc-4e5e-9e70-e17e694145ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:06:18.583843 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848c3f6b-01dc-4e5e-9e70-e17e694145ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:06:18.584437 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848c3f6b-01dc-4e5e-9e70-e17e694145ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:06:18.584781 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848c3f6b-01dc-4e5e-9e70-e17e694145ef admin admin] PATH_INFO: `/v3/users/a7093cbd398e4ba7ae8a4bb2604f022c` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:06:18.586049 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848c3f6b-01dc-4e5e-9e70-e17e694145ef admin admin] RBAC: Authorizing `identity:delete_user(user_id=a7093cbd398e4ba7ae8a4bb2604f022c)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:06:18.588402 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848c3f6b-01dc-4e5e-9e70-e17e694145ef admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:06:18.647032 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-848c3f6b-01dc-4e5e-9e70-e17e694145ef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a7093cbd398e4ba7ae8a4bb2604f022c'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:06:18.658921 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-848c3f6b-01dc-4e5e-9e70-e17e694145ef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a7093cbd398e4ba7ae8a4bb2604f022c'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:06:18.668834 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-848c3f6b-01dc-4e5e-9e70-e17e694145ef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a7093cbd398e4ba7ae8a4bb2604f022c'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:06:18.669548 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-848c3f6b-01dc-4e5e-9e70-e17e694145ef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a7093cbd398e4ba7ae8a4bb2604f022c'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:06:18.676638 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-848c3f6b-01dc-4e5e-9e70-e17e694145ef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a7093cbd398e4ba7ae8a4bb2604f022c'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:06:18.682721 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1371/2716] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Aug 30 16:06:18 2022] DELETE /identity/v3/users/a7093cbd398e4ba7ae8a4bb2604f022c => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:06:18.965135 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4f78f1-45b3-4a4d-9513-3c428fff8dd5 admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:06:18.966741 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-2e4f78f1-45b3-4a4d-9513-3c428fff8dd5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:06:18.981206 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4f78f1-45b3-4a4d-9513-3c428fff8dd5 admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:06:19.038655 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd3c5cf-0afb-4bea-a23b-c4dc794aa9cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:06:19.039203 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd3c5cf-0afb-4bea-a23b-c4dc794aa9cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:06:19.039479 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd3c5cf-0afb-4bea-a23b-c4dc794aa9cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:06:19.039749 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd3c5cf-0afb-4bea-a23b-c4dc794aa9cf admin admin] PATH_INFO: `/v3/projects/9806a12f33c64782bf5038d3e4908440` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:06:19.040902 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd3c5cf-0afb-4bea-a23b-c4dc794aa9cf admin admin] RBAC: Authorizing `identity:delete_project(project_id=9806a12f33c64782bf5038d3e4908440)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:06:19.042873 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd3c5cf-0afb-4bea-a23b-c4dc794aa9cf admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:06:19.090339 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-ccd3c5cf-0afb-4bea-a23b-c4dc794aa9cf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9806a12f33c64782bf5038d3e4908440', 'request_id': 'req-ccd3c5cf-0afb-4bea-a23b-c4dc794aa9cf'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:06:19.099449 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-ccd3c5cf-0afb-4bea-a23b-c4dc794aa9cf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9806a12f33c64782bf5038d3e4908440', 'request_id': 'req-ccd3c5cf-0afb-4bea-a23b-c4dc794aa9cf'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:06:19.106236 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-ccd3c5cf-0afb-4bea-a23b-c4dc794aa9cf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9806a12f33c64782bf5038d3e4908440', 'request_id': 'req-ccd3c5cf-0afb-4bea-a23b-c4dc794aa9cf'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:06:19.117067 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-ccd3c5cf-0afb-4bea-a23b-c4dc794aa9cf admin admin] The token cache is being invalidate because project 9806a12f33c64782bf5038d3e4908440 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81512) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 16:06:19.117476 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-ccd3c5cf-0afb-4bea-a23b-c4dc794aa9cf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:06:19.118197 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-ccd3c5cf-0afb-4bea-a23b-c4dc794aa9cf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:06:19.119903 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1346/2717] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Aug 30 16:06:18 2022] DELETE /identity/v3/projects/9806a12f33c64782bf5038d3e4908440 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:06:19.125290 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b429466b-478b-49e6-b9c1-65da25d35f11 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:06:19.125879 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b429466b-478b-49e6-b9c1-65da25d35f11 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:06:19.125879 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b429466b-478b-49e6-b9c1-65da25d35f11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:06:19.147126 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-b429466b-478b-49e6-b9c1-65da25d35f11 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:06:19.148321 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-b429466b-478b-49e6-b9c1-65da25d35f11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:06:19.219995 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1372/2718] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 16:06:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:06:19.228912 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ebc815-f705-49bc-a965-88fd8424ee26 None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:06:19.229250 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ebc815-f705-49bc-a965-88fd8424ee26 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:06:19.229632 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ebc815-f705-49bc-a965-88fd8424ee26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:06:19.257241 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-e5ebc815-f705-49bc-a965-88fd8424ee26 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:06:19.258537 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-e5ebc815-f705-49bc-a965-88fd8424ee26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:06:19.267917 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1347/2719] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 16:06:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:06:19.272404 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b429466b-478b-49e6-b9c1-65da25d35f11 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:06:19.282373 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b429466b-478b-49e6-b9c1-65da25d35f11 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:06:19.287083 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5778f92-3737-4643-a40c-4f881cbee3c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:06:19.287687 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5778f92-3737-4643-a40c-4f881cbee3c7 admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:06:19.288018 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5778f92-3737-4643-a40c-4f881cbee3c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:06:19.288287 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5778f92-3737-4643-a40c-4f881cbee3c7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:06:19.288999 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5778f92-3737-4643-a40c-4f881cbee3c7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:06:19.289264 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5778f92-3737-4643-a40c-4f881cbee3c7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:06:19.290887 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5778f92-3737-4643-a40c-4f881cbee3c7 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:06:19.303570 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1373/2720] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Aug 30 16:06:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:06:19.307789 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ebc815-f705-49bc-a965-88fd8424ee26 None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:06:19.317409 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ebc815-f705-49bc-a965-88fd8424ee26 None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:06:19.324236 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195bea4c-1ded-4680-81ed-411d82b51f31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:06:19.325419 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195bea4c-1ded-4680-81ed-411d82b51f31 admin admin] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:06:19.325419 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195bea4c-1ded-4680-81ed-411d82b51f31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:06:19.325740 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195bea4c-1ded-4680-81ed-411d82b51f31 admin admin] PATH_INFO: `/v3/projects` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:06:19.326318 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195bea4c-1ded-4680-81ed-411d82b51f31 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:06:19.328292 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195bea4c-1ded-4680-81ed-411d82b51f31 admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:06:19.358801 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1348/2721] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Aug 30 16:06:19 2022] POST /identity/v3/projects => generated 385 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:06:19.363238 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5778f92-3737-4643-a40c-4f881cbee3c7 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:06:19.375425 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5778f92-3737-4643-a40c-4f881cbee3c7 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:06:19.431713 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2c6434-45b2-4940-9f21-676247903850 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:06:19.432335 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2c6434-45b2-4940-9f21-676247903850 admin admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:06:19.432601 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2c6434-45b2-4940-9f21-676247903850 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:06:19.432868 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2c6434-45b2-4940-9f21-676247903850 admin admin] PATH_INFO: `/v3/users` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:06:19.433559 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2c6434-45b2-4940-9f21-676247903850 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:06:19.435478 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2c6434-45b2-4940-9f21-676247903850 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:06:19.446581 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:06:19.464484 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1374/2722] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Aug 30 16:06:19 2022] POST /identity/v3/users => generated 293 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 16:06:19.469157 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195bea4c-1ded-4680-81ed-411d82b51f31 admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:06:19.480352 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195bea4c-1ded-4680-81ed-411d82b51f31 admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:06:19.486516 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647fcf07-a2da-4db4-b479-84aefa7c5aa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:06:19.487107 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647fcf07-a2da-4db4-b479-84aefa7c5aa9 admin admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:06:19.487379 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647fcf07-a2da-4db4-b479-84aefa7c5aa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:06:19.487644 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647fcf07-a2da-4db4-b479-84aefa7c5aa9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:06:19.488272 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647fcf07-a2da-4db4-b479-84aefa7c5aa9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:06:19.489910 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647fcf07-a2da-4db4-b479-84aefa7c5aa9 admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:06:19.495578 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:DELETE=14,keystone:SELECT=83,keystone:INSERT=3 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:06:19.503058 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1349/2723] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 30 16:06:19 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:06:19.507758 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2c6434-45b2-4940-9f21-676247903850 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:06:19.521609 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2c6434-45b2-4940-9f21-676247903850 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:06:19.530706 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374ef845-94d5-4eb8-9a52-87d0d299a1c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:06:19.531359 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374ef845-94d5-4eb8-9a52-87d0d299a1c7 admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:06:19.531633 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374ef845-94d5-4eb8-9a52-87d0d299a1c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:06:19.531899 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374ef845-94d5-4eb8-9a52-87d0d299a1c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:06:19.532551 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374ef845-94d5-4eb8-9a52-87d0d299a1c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:06:19.534238 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374ef845-94d5-4eb8-9a52-87d0d299a1c7 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:06:19.545880 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1375/2724] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 30 16:06:19 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:06:19.550700 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647fcf07-a2da-4db4-b479-84aefa7c5aa9 admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:06:19.561661 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647fcf07-a2da-4db4-b479-84aefa7c5aa9 admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:06:19.566590 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6521561a-189e-4fcf-b8d9-3f38c1cf1274 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:06:19.567258 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6521561a-189e-4fcf-b8d9-3f38c1cf1274 admin admin] REQUEST_METHOD: `PUT` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:06:19.567534 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6521561a-189e-4fcf-b8d9-3f38c1cf1274 admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:06:19.567790 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6521561a-189e-4fcf-b8d9-3f38c1cf1274 admin admin] PATH_INFO: `/v3/projects/57a0b4a63e74412f8bb4d39dc3bb9395/users/6ca9e55f03f64fa98c163e37fd3bc57b/roles/df8f9f844b1b439181d75a07c1a93418` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:06:19.583218 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6521561a-189e-4fcf-b8d9-3f38c1cf1274 admin admin] RBAC: Authorizing `identity:create_grant(project_id=57a0b4a63e74412f8bb4d39dc3bb9395, user_id=6ca9e55f03f64fa98c163e37fd3bc57b, role_id=df8f9f844b1b439181d75a07c1a93418)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:06:19.585942 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6521561a-189e-4fcf-b8d9-3f38c1cf1274 admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:06:19.600243 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1350/2725] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 30 16:06:19 2022] PUT /identity/v3/projects/57a0b4a63e74412f8bb4d39dc3bb9395/users/6ca9e55f03f64fa98c163e37fd3bc57b/roles/df8f9f844b1b439181d75a07c1a93418 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:06:19.611139 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972a4342-d018-4b3b-9588-7426514eeef3 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:06:19.611421 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972a4342-d018-4b3b-9588-7426514eeef3 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:06:19.611688 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972a4342-d018-4b3b-9588-7426514eeef3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:06:19.641226 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-972a4342-d018-4b3b-9588-7426514eeef3 None None] MFA Rules not processed for user `6ca9e55f03f64fa98c163e37fd3bc57b`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:06:19.642525 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-972a4342-d018-4b3b-9588-7426514eeef3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:06:19.710070 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1376/2726] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 16:06:19 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 16:06:19.723058 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6521561a-189e-4fcf-b8d9-3f38c1cf1274 admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:06:19.725132 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-6521561a-189e-4fcf-b8d9-3f38c1cf1274 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:06:19.791009 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6521561a-189e-4fcf-b8d9-3f38c1cf1274 admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:06:19.796992 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf692df-8a14-4432-9283-69195f891c1c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0100a04821804993866ea20f159663f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:06:19.797787 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf692df-8a14-4432-9283-69195f891c1c service cinder] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:06:19.798158 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf692df-8a14-4432-9283-69195f891c1c service cinder] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:06:19.798419 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf692df-8a14-4432-9283-69195f891c1c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:06:19.808811 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf692df-8a14-4432-9283-69195f891c1c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:06:19.810283 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf692df-8a14-4432-9283-69195f891c1c service cinder] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:06:19.819161 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1351/2727] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 16:06:19 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:06:19.844309 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972a4342-d018-4b3b-9588-7426514eeef3 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:06:19.907130 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972a4342-d018-4b3b-9588-7426514eeef3 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:06:19.912135 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec29776e-3578-4710-bf4b-208928e0c0f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:06:19.912757 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec29776e-3578-4710-bf4b-208928e0c0f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:06:19.913035 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec29776e-3578-4710-bf4b-208928e0c0f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:06:19.913396 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec29776e-3578-4710-bf4b-208928e0c0f2 admin admin] PATH_INFO: `/v3/users/6ca9e55f03f64fa98c163e37fd3bc57b` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:06:19.914596 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec29776e-3578-4710-bf4b-208928e0c0f2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6ca9e55f03f64fa98c163e37fd3bc57b)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:06:19.916507 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec29776e-3578-4710-bf4b-208928e0c0f2 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:06:19.973929 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-ec29776e-3578-4710-bf4b-208928e0c0f2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6ca9e55f03f64fa98c163e37fd3bc57b'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:06:19.990005 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-ec29776e-3578-4710-bf4b-208928e0c0f2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6ca9e55f03f64fa98c163e37fd3bc57b'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:06:20.001689 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-ec29776e-3578-4710-bf4b-208928e0c0f2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6ca9e55f03f64fa98c163e37fd3bc57b'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:06:20.002649 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-ec29776e-3578-4710-bf4b-208928e0c0f2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6ca9e55f03f64fa98c163e37fd3bc57b'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:06:20.005728 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:INSERT=2,keystone:SELECT=88,keystone:DELETE=10 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:06:20.012161 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-ec29776e-3578-4710-bf4b-208928e0c0f2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6ca9e55f03f64fa98c163e37fd3bc57b'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:06:20.018873 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1377/2728] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Aug 30 16:06:19 2022] DELETE /identity/v3/users/6ca9e55f03f64fa98c163e37fd3bc57b => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:06:20.023508 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a7adc8-0941-4530-af7d-7de29a2de72d None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:06:20.024002 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a7adc8-0941-4530-af7d-7de29a2de72d None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:06:20.024099 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a7adc8-0941-4530-af7d-7de29a2de72d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:06:20.044062 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-86a7adc8-0941-4530-af7d-7de29a2de72d None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:06:20.045293 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-86a7adc8-0941-4530-af7d-7de29a2de72d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:06:20.110640 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1352/2729] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 16:06:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 16:06:20.124930 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec29776e-3578-4710-bf4b-208928e0c0f2 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:06:20.126968 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-ec29776e-3578-4710-bf4b-208928e0c0f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:06:20.199635 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec29776e-3578-4710-bf4b-208928e0c0f2 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:06:20.205486 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82755fd7-c800-4985-a054-1d69ccaf0d4b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f4157105f3488f807b7475008ab722', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:06:20.207712 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82755fd7-c800-4985-a054-1d69ccaf0d4b service neutron] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:06:20.207995 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82755fd7-c800-4985-a054-1d69ccaf0d4b service neutron] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:06:20.208266 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82755fd7-c800-4985-a054-1d69ccaf0d4b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:06:20.218736 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82755fd7-c800-4985-a054-1d69ccaf0d4b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:06:20.220183 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82755fd7-c800-4985-a054-1d69ccaf0d4b service neutron] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:06:20.229767 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1378/2730] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 16:06:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:06:20.480351 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a7adc8-0941-4530-af7d-7de29a2de72d None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:06:20.490788 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a7adc8-0941-4530-af7d-7de29a2de72d None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:06:20.494603 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f43546-0569-4ce5-a681-aac5075524d4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f4157105f3488f807b7475008ab722', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:06:20.495430 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f43546-0569-4ce5-a681-aac5075524d4 service neutron] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:06:20.495705 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f43546-0569-4ce5-a681-aac5075524d4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:06:20.495979 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f43546-0569-4ce5-a681-aac5075524d4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:06:20.497743 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-a5f43546-0569-4ce5-a681-aac5075524d4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:06:20.510506 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f43546-0569-4ce5-a681-aac5075524d4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:06:20.511944 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f43546-0569-4ce5-a681-aac5075524d4 service neutron] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:06:20.521626 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82755fd7-c800-4985-a054-1d69ccaf0d4b service neutron] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:06:20.522122 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1353/2731] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 16:06:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:06:20.523158 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-82755fd7-c800-4985-a054-1d69ccaf0d4b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:06:20.538108 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82755fd7-c800-4985-a054-1d69ccaf0d4b service neutron] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:06:20.543608 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546dc6c7-89d1-4f81-b744-ec750a80a641 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:06:20.544258 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546dc6c7-89d1-4f81-b744-ec750a80a641 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:06:20.544533 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546dc6c7-89d1-4f81-b744-ec750a80a641 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:06:20.544830 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546dc6c7-89d1-4f81-b744-ec750a80a641 admin admin] PATH_INFO: `/v3/projects/57a0b4a63e74412f8bb4d39dc3bb9395` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:06:20.546095 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546dc6c7-89d1-4f81-b744-ec750a80a641 admin admin] RBAC: Authorizing `identity:delete_project(project_id=57a0b4a63e74412f8bb4d39dc3bb9395)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:06:20.548026 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546dc6c7-89d1-4f81-b744-ec750a80a641 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:06:20.597660 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-546dc6c7-89d1-4f81-b744-ec750a80a641 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '57a0b4a63e74412f8bb4d39dc3bb9395', 'request_id': 'req-546dc6c7-89d1-4f81-b744-ec750a80a641'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:06:20.607075 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-546dc6c7-89d1-4f81-b744-ec750a80a641 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '57a0b4a63e74412f8bb4d39dc3bb9395', 'request_id': 'req-546dc6c7-89d1-4f81-b744-ec750a80a641'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:06:20.614238 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-546dc6c7-89d1-4f81-b744-ec750a80a641 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '57a0b4a63e74412f8bb4d39dc3bb9395', 'request_id': 'req-546dc6c7-89d1-4f81-b744-ec750a80a641'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:06:20.624924 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-546dc6c7-89d1-4f81-b744-ec750a80a641 admin admin] The token cache is being invalidate because project 57a0b4a63e74412f8bb4d39dc3bb9395 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81511) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 16:06:20.625196 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-546dc6c7-89d1-4f81-b744-ec750a80a641 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:06:20.626299 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-546dc6c7-89d1-4f81-b744-ec750a80a641 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:06:20.631809 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1379/2732] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Aug 30 16:06:20 2022] DELETE /identity/v3/projects/57a0b4a63e74412f8bb4d39dc3bb9395 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 16:06:20.633500 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabe8d15-7608-4a56-a680-e9e3da982c57 None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:06:20.633766 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabe8d15-7608-4a56-a680-e9e3da982c57 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:06:20.634037 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabe8d15-7608-4a56-a680-e9e3da982c57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:06:20.659115 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-eabe8d15-7608-4a56-a680-e9e3da982c57 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:06:20.660326 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-eabe8d15-7608-4a56-a680-e9e3da982c57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:06:20.719807 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1354/2733] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 16:06:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 16:06:20.727677 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c6dd21-dccc-47cc-8728-9d438cbceda3 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:06:20.727961 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c6dd21-dccc-47cc-8728-9d438cbceda3 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:06:20.728244 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c6dd21-dccc-47cc-8728-9d438cbceda3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:06:20.748365 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-a3c6dd21-dccc-47cc-8728-9d438cbceda3 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:06:20.749614 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-a3c6dd21-dccc-47cc-8728-9d438cbceda3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:06:20.757695 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1380/2734] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 16:06:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:06:20.762310 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabe8d15-7608-4a56-a680-e9e3da982c57 None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:06:20.772642 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabe8d15-7608-4a56-a680-e9e3da982c57 None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:06:20.777621 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0a84fd-9373-4858-94d1-bdb102cc010c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:06:20.778318 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0a84fd-9373-4858-94d1-bdb102cc010c admin admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:06:20.778595 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0a84fd-9373-4858-94d1-bdb102cc010c admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:06:20.778861 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0a84fd-9373-4858-94d1-bdb102cc010c admin admin] PATH_INFO: `/v3/domains` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:06:20.779562 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0a84fd-9373-4858-94d1-bdb102cc010c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81512) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:06:20.779863 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0a84fd-9373-4858-94d1-bdb102cc010c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:06:20.781547 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0a84fd-9373-4858-94d1-bdb102cc010c admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:06:20.783321 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:06:20.796343 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1355/2735] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Aug 30 16:06:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 16:06:20.800791 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c6dd21-dccc-47cc-8728-9d438cbceda3 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:06:20.810662 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c6dd21-dccc-47cc-8728-9d438cbceda3 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:06:20.815614 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08fc9f2-073a-42ef-916d-27a47c83eaf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:06:20.816242 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08fc9f2-073a-42ef-916d-27a47c83eaf9 admin admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:06:20.816515 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08fc9f2-073a-42ef-916d-27a47c83eaf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:06:20.816786 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08fc9f2-073a-42ef-916d-27a47c83eaf9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:06:20.817469 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08fc9f2-073a-42ef-916d-27a47c83eaf9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:06:20.819605 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08fc9f2-073a-42ef-916d-27a47c83eaf9 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:06:20.854776 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1381/2736] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Aug 30 16:06:20 2022] POST /identity/v3/projects => generated 383 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:06:20.856500 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0a84fd-9373-4858-94d1-bdb102cc010c admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:06:20.867357 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0a84fd-9373-4858-94d1-bdb102cc010c admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:06:20.928979 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f23ca4-9fc2-4edc-81b0-70d4dbcbd11f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:06:20.929541 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f23ca4-9fc2-4edc-81b0-70d4dbcbd11f admin admin] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:06:20.929826 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f23ca4-9fc2-4edc-81b0-70d4dbcbd11f admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:06:20.930097 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f23ca4-9fc2-4edc-81b0-70d4dbcbd11f admin admin] PATH_INFO: `/v3/users` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:06:20.930740 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f23ca4-9fc2-4edc-81b0-70d4dbcbd11f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:06:20.932677 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f23ca4-9fc2-4edc-81b0-70d4dbcbd11f admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:06:20.956304 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1356/2737] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Aug 30 16:06:20 2022] POST /identity/v3/users => generated 292 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:06:20.960701 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08fc9f2-073a-42ef-916d-27a47c83eaf9 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:06:20.972144 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08fc9f2-073a-42ef-916d-27a47c83eaf9 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:06:20.976840 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b29ac0-0d60-4552-a0e4-efa08e24e00f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:06:20.977484 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b29ac0-0d60-4552-a0e4-efa08e24e00f admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:06:20.977764 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b29ac0-0d60-4552-a0e4-efa08e24e00f admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:06:20.978041 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b29ac0-0d60-4552-a0e4-efa08e24e00f admin admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:06:20.978700 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b29ac0-0d60-4552-a0e4-efa08e24e00f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:06:20.980295 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b29ac0-0d60-4552-a0e4-efa08e24e00f admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:06:20.991800 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1382/2738] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 30 16:06:20 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:06:20.996132 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f23ca4-9fc2-4edc-81b0-70d4dbcbd11f admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:06:21.007304 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f23ca4-9fc2-4edc-81b0-70d4dbcbd11f admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:06:21.013015 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28280f1e-a4cd-40bf-a99e-7269f165d9f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:06:21.013729 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28280f1e-a4cd-40bf-a99e-7269f165d9f1 admin admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:06:21.014008 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28280f1e-a4cd-40bf-a99e-7269f165d9f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:06:21.014276 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28280f1e-a4cd-40bf-a99e-7269f165d9f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:06:21.014926 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28280f1e-a4cd-40bf-a99e-7269f165d9f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:06:21.016534 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28280f1e-a4cd-40bf-a99e-7269f165d9f1 admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:06:21.028035 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1357/2739] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 30 16:06:20 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 16:06:21.032307 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b29ac0-0d60-4552-a0e4-efa08e24e00f admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:06:21.043088 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b29ac0-0d60-4552-a0e4-efa08e24e00f admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:06:21.047681 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0c9186-2822-481c-91f9-cbb3c841343c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:06:21.048335 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0c9186-2822-481c-91f9-cbb3c841343c admin admin] REQUEST_METHOD: `PUT` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:06:21.048610 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0c9186-2822-481c-91f9-cbb3c841343c admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:06:21.048879 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0c9186-2822-481c-91f9-cbb3c841343c admin admin] PATH_INFO: `/v3/projects/86ce8e5df7294b60aa9b870e1295a1f0/users/11debfd5eb154a1facb01143296e3fd8/roles/df8f9f844b1b439181d75a07c1a93418` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:06:21.065167 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0c9186-2822-481c-91f9-cbb3c841343c admin admin] RBAC: Authorizing `identity:create_grant(project_id=86ce8e5df7294b60aa9b870e1295a1f0, user_id=11debfd5eb154a1facb01143296e3fd8, role_id=df8f9f844b1b439181d75a07c1a93418)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:06:21.068117 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0c9186-2822-481c-91f9-cbb3c841343c admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:06:21.082894 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1383/2740] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 30 16:06:21 2022] PUT /identity/v3/projects/86ce8e5df7294b60aa9b870e1295a1f0/users/11debfd5eb154a1facb01143296e3fd8/roles/df8f9f844b1b439181d75a07c1a93418 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:06:21.156911 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6b285c-4225-4c93-9141-f1f272651894 None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:06:21.157199 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6b285c-4225-4c93-9141-f1f272651894 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:06:21.157494 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6b285c-4225-4c93-9141-f1f272651894 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:06:21.187488 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-ef6b285c-4225-4c93-9141-f1f272651894 None None] MFA Rules not processed for user `11debfd5eb154a1facb01143296e3fd8`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:06:21.188795 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-ef6b285c-4225-4c93-9141-f1f272651894 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:06:21.254788 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1358/2741] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 16:06:21 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:06:21.266363 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0c9186-2822-481c-91f9-cbb3c841343c admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:06:21.268593 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-cd0c9186-2822-481c-91f9-cbb3c841343c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:06:21.283092 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:06:21.327433 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0c9186-2822-481c-91f9-cbb3c841343c admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:06:21.331443 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8e29eb-f822-4e67-894f-63fb475e5675 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0100a04821804993866ea20f159663f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:06:21.331792 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8e29eb-f822-4e67-894f-63fb475e5675 service cinder] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:06:21.332064 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8e29eb-f822-4e67-894f-63fb475e5675 service cinder] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:06:21.332330 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8e29eb-f822-4e67-894f-63fb475e5675 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:06:21.340990 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8e29eb-f822-4e67-894f-63fb475e5675 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:06:21.342469 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8e29eb-f822-4e67-894f-63fb475e5675 service cinder] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:06:21.352734 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1384/2742] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 16:06:21 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:06:23.735170 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6b285c-4225-4c93-9141-f1f272651894 None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:06:23.739287 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-ef6b285c-4225-4c93-9141-f1f272651894 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:06:23.815666 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8e29eb-f822-4e67-894f-63fb475e5675 service cinder] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:06:23.817359 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-9c8e29eb-f822-4e67-894f-63fb475e5675 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:06:23.859935 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:06:23.865804 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6b285c-4225-4c93-9141-f1f272651894 None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:06:23.872904 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdfc544-f978-4096-a964-9bd8cc93cf13 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc12c6d31eab4a17a160250adc2380c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:06:23.875198 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdfc544-f978-4096-a964-9bd8cc93cf13 service placement] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:06:23.876448 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdfc544-f978-4096-a964-9bd8cc93cf13 service placement] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:06:23.876815 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdfc544-f978-4096-a964-9bd8cc93cf13 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:06:23.879165 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-dfdfc544-f978-4096-a964-9bd8cc93cf13 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:06:23.893184 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdfc544-f978-4096-a964-9bd8cc93cf13 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:06:23.896293 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdfc544-f978-4096-a964-9bd8cc93cf13 service placement] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:06:23.908469 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1359/2743] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 16:06:23 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:06:23.917948 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8e29eb-f822-4e67-894f-63fb475e5675 service cinder] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:06:23.922791 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06071dda-f9ae-462b-8531-65e8e562dcca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:06:23.923494 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06071dda-f9ae-462b-8531-65e8e562dcca admin admin] REQUEST_METHOD: `DELETE` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:06:23.923838 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06071dda-f9ae-462b-8531-65e8e562dcca admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:06:23.924174 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06071dda-f9ae-462b-8531-65e8e562dcca admin admin] PATH_INFO: `/v3/users/252dd6215082420cade91b100c73f100` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:06:23.925514 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06071dda-f9ae-462b-8531-65e8e562dcca admin admin] RBAC: Authorizing `identity:delete_user(user_id=252dd6215082420cade91b100c73f100)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:06:23.927575 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06071dda-f9ae-462b-8531-65e8e562dcca admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:06:23.982066 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-06071dda-f9ae-462b-8531-65e8e562dcca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '252dd6215082420cade91b100c73f100'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:06:23.994217 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-06071dda-f9ae-462b-8531-65e8e562dcca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '252dd6215082420cade91b100c73f100'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:06:24.003527 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-06071dda-f9ae-462b-8531-65e8e562dcca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '252dd6215082420cade91b100c73f100'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:06:24.004322 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-06071dda-f9ae-462b-8531-65e8e562dcca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '252dd6215082420cade91b100c73f100'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:06:24.011326 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-06071dda-f9ae-462b-8531-65e8e562dcca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '252dd6215082420cade91b100c73f100'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:06:24.017132 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1385/2744] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Aug 30 16:06:23 2022] DELETE /identity/v3/users/252dd6215082420cade91b100c73f100 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 16:06:24.201154 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdfc544-f978-4096-a964-9bd8cc93cf13 service placement] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:06:24.216607 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdfc544-f978-4096-a964-9bd8cc93cf13 service placement] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:06:24.274724 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45b4d28-52a2-4d74-b5f6-9fb80574843e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:06:24.275387 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45b4d28-52a2-4d74-b5f6-9fb80574843e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:06:24.275662 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45b4d28-52a2-4d74-b5f6-9fb80574843e admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:06:24.275930 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45b4d28-52a2-4d74-b5f6-9fb80574843e admin admin] PATH_INFO: `/v3/projects/6550878193fa4088861f08552a276578` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:06:24.277066 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45b4d28-52a2-4d74-b5f6-9fb80574843e admin admin] RBAC: Authorizing `identity:delete_project(project_id=6550878193fa4088861f08552a276578)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:06:24.279010 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45b4d28-52a2-4d74-b5f6-9fb80574843e admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:06:24.332510 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-d45b4d28-52a2-4d74-b5f6-9fb80574843e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6550878193fa4088861f08552a276578', 'request_id': 'req-d45b4d28-52a2-4d74-b5f6-9fb80574843e'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:06:24.342196 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-d45b4d28-52a2-4d74-b5f6-9fb80574843e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6550878193fa4088861f08552a276578', 'request_id': 'req-d45b4d28-52a2-4d74-b5f6-9fb80574843e'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:06:24.348835 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-d45b4d28-52a2-4d74-b5f6-9fb80574843e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6550878193fa4088861f08552a276578', 'request_id': 'req-d45b4d28-52a2-4d74-b5f6-9fb80574843e'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:06:24.360817 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-d45b4d28-52a2-4d74-b5f6-9fb80574843e admin admin] The token cache is being invalidate because project 6550878193fa4088861f08552a276578 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81512) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 16:06:24.361104 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-d45b4d28-52a2-4d74-b5f6-9fb80574843e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:06:24.361990 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-d45b4d28-52a2-4d74-b5f6-9fb80574843e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:06:24.363734 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1360/2745] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Aug 30 16:06:24 2022] DELETE /identity/v3/projects/6550878193fa4088861f08552a276578 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:06:24.368822 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193b6f01-e36c-4d1c-9fbf-e82e176da6ba None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:06:24.369196 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193b6f01-e36c-4d1c-9fbf-e82e176da6ba None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:06:24.369527 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193b6f01-e36c-4d1c-9fbf-e82e176da6ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:06:24.389023 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-193b6f01-e36c-4d1c-9fbf-e82e176da6ba None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:06:24.390307 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-193b6f01-e36c-4d1c-9fbf-e82e176da6ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:06:24.421117 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:06:24.455745 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1386/2746] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 16:06:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:06:24.463557 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da03229-a7ff-43fc-8adf-5a38d33483d0 None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:06:24.463840 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da03229-a7ff-43fc-8adf-5a38d33483d0 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:06:24.464111 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da03229-a7ff-43fc-8adf-5a38d33483d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:06:24.484611 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-6da03229-a7ff-43fc-8adf-5a38d33483d0 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:06:24.485867 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-6da03229-a7ff-43fc-8adf-5a38d33483d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:06:24.494348 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1361/2747] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 16:06:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:06:24.498476 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193b6f01-e36c-4d1c-9fbf-e82e176da6ba None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:06:24.508507 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193b6f01-e36c-4d1c-9fbf-e82e176da6ba None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:06:24.513766 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5973f645-1526-4b40-844f-8c431e6ea8ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:06:24.514362 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5973f645-1526-4b40-844f-8c431e6ea8ab admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:06:24.514638 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5973f645-1526-4b40-844f-8c431e6ea8ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:06:24.514904 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5973f645-1526-4b40-844f-8c431e6ea8ab admin admin] PATH_INFO: `/v3/domains` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:06:24.515582 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5973f645-1526-4b40-844f-8c431e6ea8ab admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:06:24.515881 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5973f645-1526-4b40-844f-8c431e6ea8ab admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:06:24.517510 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5973f645-1526-4b40-844f-8c431e6ea8ab admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:06:24.529733 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1387/2748] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Aug 30 16:06:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:06:24.534141 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da03229-a7ff-43fc-8adf-5a38d33483d0 None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:06:24.545781 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da03229-a7ff-43fc-8adf-5a38d33483d0 None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:06:24.553874 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4000e4-33a9-49ca-8c5b-71fb5d990fd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:06:24.554475 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4000e4-33a9-49ca-8c5b-71fb5d990fd9 admin admin] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:06:24.554896 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4000e4-33a9-49ca-8c5b-71fb5d990fd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:06:24.555301 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4000e4-33a9-49ca-8c5b-71fb5d990fd9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:06:24.556263 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4000e4-33a9-49ca-8c5b-71fb5d990fd9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:06:24.559044 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4000e4-33a9-49ca-8c5b-71fb5d990fd9 admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:06:24.594737 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1362/2749] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Aug 30 16:06:24 2022] POST /identity/v3/projects => generated 377 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:06:24.599394 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5973f645-1526-4b40-844f-8c431e6ea8ab admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:06:24.610355 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5973f645-1526-4b40-844f-8c431e6ea8ab admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:06:24.670502 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d134402-5fd3-4982-859b-82b971956d68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:06:24.671117 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d134402-5fd3-4982-859b-82b971956d68 admin admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:06:24.671398 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d134402-5fd3-4982-859b-82b971956d68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:06:24.671672 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d134402-5fd3-4982-859b-82b971956d68 admin admin] PATH_INFO: `/v3/users` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:06:24.672307 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d134402-5fd3-4982-859b-82b971956d68 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:06:24.674300 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d134402-5fd3-4982-859b-82b971956d68 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:06:24.698214 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1388/2750] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Aug 30 16:06:24 2022] POST /identity/v3/users => generated 289 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:06:24.702573 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4000e4-33a9-49ca-8c5b-71fb5d990fd9 admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:06:24.713027 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4000e4-33a9-49ca-8c5b-71fb5d990fd9 admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:06:24.718247 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad8578b-10e5-4637-abc2-2d0d8616e118 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:06:24.718893 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad8578b-10e5-4637-abc2-2d0d8616e118 admin admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:06:24.719154 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad8578b-10e5-4637-abc2-2d0d8616e118 admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:06:24.719422 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad8578b-10e5-4637-abc2-2d0d8616e118 admin admin] PATH_INFO: `/v3/roles` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:06:24.720064 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad8578b-10e5-4637-abc2-2d0d8616e118 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:06:24.721712 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad8578b-10e5-4637-abc2-2d0d8616e118 admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:06:24.732048 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1363/2751] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 30 16:06:24 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 16:06:24.736270 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d134402-5fd3-4982-859b-82b971956d68 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:06:24.746990 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d134402-5fd3-4982-859b-82b971956d68 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:06:24.751882 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279b49de-4abb-416e-8e79-b48dd38583b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:06:24.752490 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279b49de-4abb-416e-8e79-b48dd38583b2 admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:06:24.752766 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279b49de-4abb-416e-8e79-b48dd38583b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:06:24.753038 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279b49de-4abb-416e-8e79-b48dd38583b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:06:24.753741 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279b49de-4abb-416e-8e79-b48dd38583b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:06:24.755328 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279b49de-4abb-416e-8e79-b48dd38583b2 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:06:24.766127 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1389/2752] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 30 16:06:24 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:06:24.770471 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad8578b-10e5-4637-abc2-2d0d8616e118 admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:06:24.780689 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad8578b-10e5-4637-abc2-2d0d8616e118 admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:06:24.786389 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563462a0-5635-4f57-b772-5df0f0b4ce72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:06:24.787026 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563462a0-5635-4f57-b772-5df0f0b4ce72 admin admin] REQUEST_METHOD: `PUT` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:06:24.787303 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563462a0-5635-4f57-b772-5df0f0b4ce72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:06:24.787572 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563462a0-5635-4f57-b772-5df0f0b4ce72 admin admin] PATH_INFO: `/v3/projects/12d8ded5bfda4c8ebb281a5504b56eb1/users/1cd3f1cbe0eb496f8e4c0692366fdefb/roles/df8f9f844b1b439181d75a07c1a93418` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:06:24.803401 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563462a0-5635-4f57-b772-5df0f0b4ce72 admin admin] RBAC: Authorizing `identity:create_grant(project_id=12d8ded5bfda4c8ebb281a5504b56eb1, user_id=1cd3f1cbe0eb496f8e4c0692366fdefb, role_id=df8f9f844b1b439181d75a07c1a93418)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:06:24.806119 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563462a0-5635-4f57-b772-5df0f0b4ce72 admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:06:24.822149 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1364/2753] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 30 16:06:24 2022] PUT /identity/v3/projects/12d8ded5bfda4c8ebb281a5504b56eb1/users/1cd3f1cbe0eb496f8e4c0692366fdefb/roles/df8f9f844b1b439181d75a07c1a93418 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:06:24.827219 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8228ee-dcf6-4ca1-b016-2b51de3e65b0 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:06:24.827626 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8228ee-dcf6-4ca1-b016-2b51de3e65b0 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:06:24.827719 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8228ee-dcf6-4ca1-b016-2b51de3e65b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:06:24.852975 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-ac8228ee-dcf6-4ca1-b016-2b51de3e65b0 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:06:24.854220 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-ac8228ee-dcf6-4ca1-b016-2b51de3e65b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:06:24.918419 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1390/2754] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 16:06:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:06:24.931231 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563462a0-5635-4f57-b772-5df0f0b4ce72 admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:06:24.933125 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-563462a0-5635-4f57-b772-5df0f0b4ce72 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:06:24.953807 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:06:24.992037 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563462a0-5635-4f57-b772-5df0f0b4ce72 admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:06:24.996046 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a056ed-d6ad-41cf-a29e-4fb7dea35690 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f4157105f3488f807b7475008ab722', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:06:24.996650 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a056ed-d6ad-41cf-a29e-4fb7dea35690 service neutron] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:06:24.996930 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a056ed-d6ad-41cf-a29e-4fb7dea35690 service neutron] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:06:24.997195 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a056ed-d6ad-41cf-a29e-4fb7dea35690 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:06:25.008409 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a056ed-d6ad-41cf-a29e-4fb7dea35690 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:06:25.009872 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a056ed-d6ad-41cf-a29e-4fb7dea35690 service neutron] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:06:25.019284 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1365/2755] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 16:06:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 16:06:29.061680 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23220c7c-377e-4693-a809-a59e6644166d None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:06:29.062218 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23220c7c-377e-4693-a809-a59e6644166d None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:06:29.062218 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23220c7c-377e-4693-a809-a59e6644166d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:06:29.091436 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-23220c7c-377e-4693-a809-a59e6644166d None None] MFA Rules not processed for user `1cd3f1cbe0eb496f8e4c0692366fdefb`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:06:29.092650 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-23220c7c-377e-4693-a809-a59e6644166d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:06:29.129664 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:06:29.171145 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1391/2756] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 16:06:29 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:06:29.186167 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a056ed-d6ad-41cf-a29e-4fb7dea35690 service neutron] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:06:29.187763 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-34a056ed-d6ad-41cf-a29e-4fb7dea35690 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:06:29.288680 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a056ed-d6ad-41cf-a29e-4fb7dea35690 service neutron] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:06:29.297094 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006c5cd8-6304-40bb-97c6-27886ba1d444 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a1e756ae9be4a7bac1e81acf07e8f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:06:29.299131 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006c5cd8-6304-40bb-97c6-27886ba1d444 service nova] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:06:29.299582 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006c5cd8-6304-40bb-97c6-27886ba1d444 service nova] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:06:29.299853 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006c5cd8-6304-40bb-97c6-27886ba1d444 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:06:29.308774 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006c5cd8-6304-40bb-97c6-27886ba1d444 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:06:29.310434 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006c5cd8-6304-40bb-97c6-27886ba1d444 service nova] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:06:29.319996 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1366/2757] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 16:06:29 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:06:30.938608 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6654e70a-f28f-46a5-959e-008f43aff2e5 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:06:30.939095 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6654e70a-f28f-46a5-959e-008f43aff2e5 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:06:30.939211 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6654e70a-f28f-46a5-959e-008f43aff2e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:06:30.959138 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-6654e70a-f28f-46a5-959e-008f43aff2e5 None None] MFA Rules not processed for user `1cd3f1cbe0eb496f8e4c0692366fdefb`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:06:30.960361 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-6654e70a-f28f-46a5-959e-008f43aff2e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:06:30.967741 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1392/2758] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 16:06:30 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:06:31.055033 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006c5cd8-6304-40bb-97c6-27886ba1d444 service nova] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:06:31.056643 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-006c5cd8-6304-40bb-97c6-27886ba1d444 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:06:31.071971 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006c5cd8-6304-40bb-97c6-27886ba1d444 service nova] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:06:31.077896 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1576917-806f-487d-b512-e23b84928608 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:06:31.078544 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1576917-806f-487d-b512-e23b84928608 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:06:31.078813 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1576917-806f-487d-b512-e23b84928608 admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:06:31.079078 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1576917-806f-487d-b512-e23b84928608 admin admin] PATH_INFO: `/v3/users/11debfd5eb154a1facb01143296e3fd8` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:06:31.080237 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1576917-806f-487d-b512-e23b84928608 admin admin] RBAC: Authorizing `identity:delete_user(user_id=11debfd5eb154a1facb01143296e3fd8)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:06:31.083001 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1576917-806f-487d-b512-e23b84928608 admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:06:31.155872 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-e1576917-806f-487d-b512-e23b84928608 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '11debfd5eb154a1facb01143296e3fd8'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:06:31.168059 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-e1576917-806f-487d-b512-e23b84928608 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '11debfd5eb154a1facb01143296e3fd8'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:06:31.185294 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-e1576917-806f-487d-b512-e23b84928608 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '11debfd5eb154a1facb01143296e3fd8'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:06:31.194779 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-e1576917-806f-487d-b512-e23b84928608 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '11debfd5eb154a1facb01143296e3fd8'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:06:31.195562 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-e1576917-806f-487d-b512-e23b84928608 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '11debfd5eb154a1facb01143296e3fd8'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:06:31.204547 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1367/2759] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Aug 30 16:06:31 2022] DELETE /identity/v3/users/11debfd5eb154a1facb01143296e3fd8 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:06:31.206330 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05637b04-e088-485e-a656-4447b718528a None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:06:31.206812 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05637b04-e088-485e-a656-4447b718528a None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:06:31.207181 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05637b04-e088-485e-a656-4447b718528a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:06:31.250383 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-05637b04-e088-485e-a656-4447b718528a None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:06:31.252182 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-05637b04-e088-485e-a656-4447b718528a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:06:31.324750 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1393/2760] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 16:06:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:06:31.334743 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1576917-806f-487d-b512-e23b84928608 admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:06:31.336282 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-e1576917-806f-487d-b512-e23b84928608 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:06:31.362179 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:06:31.417311 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1576917-806f-487d-b512-e23b84928608 admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:06:31.421544 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7011620e-ff97-4079-9795-6f551f5f590c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f4157105f3488f807b7475008ab722', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:06:31.422314 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7011620e-ff97-4079-9795-6f551f5f590c service neutron] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:06:31.422855 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7011620e-ff97-4079-9795-6f551f5f590c service neutron] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:06:31.423226 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7011620e-ff97-4079-9795-6f551f5f590c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:06:31.435578 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7011620e-ff97-4079-9795-6f551f5f590c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:06:31.437453 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7011620e-ff97-4079-9795-6f551f5f590c service neutron] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:06:31.450047 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1368/2761] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 16:06:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:06:31.827125 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05637b04-e088-485e-a656-4447b718528a None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:06:31.828570 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-05637b04-e088-485e-a656-4447b718528a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:06:31.848506 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05637b04-e088-485e-a656-4447b718528a None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:06:31.854770 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ddc8b4-ad74-44d2-8018-51107a218808 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:06:31.855692 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ddc8b4-ad74-44d2-8018-51107a218808 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:06:31.855967 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ddc8b4-ad74-44d2-8018-51107a218808 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:06:31.856235 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ddc8b4-ad74-44d2-8018-51107a218808 admin admin] PATH_INFO: `/v3/projects/86ce8e5df7294b60aa9b870e1295a1f0` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:06:31.857881 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ddc8b4-ad74-44d2-8018-51107a218808 admin admin] RBAC: Authorizing `identity:delete_project(project_id=86ce8e5df7294b60aa9b870e1295a1f0)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:06:31.860170 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ddc8b4-ad74-44d2-8018-51107a218808 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:06:31.924336 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-a2ddc8b4-ad74-44d2-8018-51107a218808 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '86ce8e5df7294b60aa9b870e1295a1f0', 'request_id': 'req-a2ddc8b4-ad74-44d2-8018-51107a218808'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:06:31.936612 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-a2ddc8b4-ad74-44d2-8018-51107a218808 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '86ce8e5df7294b60aa9b870e1295a1f0', 'request_id': 'req-a2ddc8b4-ad74-44d2-8018-51107a218808'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:06:31.944232 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-a2ddc8b4-ad74-44d2-8018-51107a218808 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '86ce8e5df7294b60aa9b870e1295a1f0', 'request_id': 'req-a2ddc8b4-ad74-44d2-8018-51107a218808'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:06:31.955332 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-a2ddc8b4-ad74-44d2-8018-51107a218808 admin admin] The token cache is being invalidate because project 86ce8e5df7294b60aa9b870e1295a1f0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81511) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 16:06:31.955620 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-a2ddc8b4-ad74-44d2-8018-51107a218808 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:06:31.956668 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-a2ddc8b4-ad74-44d2-8018-51107a218808 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:06:31.958170 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1394/2762] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Aug 30 16:06:31 2022] DELETE /identity/v3/projects/86ce8e5df7294b60aa9b870e1295a1f0 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:06:31.964385 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b33025-05a8-4afe-a378-bc90a147e060 None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:06:31.964857 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b33025-05a8-4afe-a378-bc90a147e060 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:06:31.965592 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b33025-05a8-4afe-a378-bc90a147e060 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:06:31.999590 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-d3b33025-05a8-4afe-a378-bc90a147e060 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:06:32.000834 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-d3b33025-05a8-4afe-a378-bc90a147e060 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:06:32.083836 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1369/2763] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 16:06:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 16:06:32.091562 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f348977-2f7f-4851-9fd9-90996b9ca1d0 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:06:32.091848 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f348977-2f7f-4851-9fd9-90996b9ca1d0 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:06:32.092121 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f348977-2f7f-4851-9fd9-90996b9ca1d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:06:32.118305 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-4f348977-2f7f-4851-9fd9-90996b9ca1d0 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:06:32.119614 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-4f348977-2f7f-4851-9fd9-90996b9ca1d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:06:32.129679 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1395/2764] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 16:06:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:06:32.135363 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b33025-05a8-4afe-a378-bc90a147e060 None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:06:32.147460 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b33025-05a8-4afe-a378-bc90a147e060 None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:06:32.154000 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99c2d62-565f-40fe-93c7-3b6447e6af3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:06:32.154962 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99c2d62-565f-40fe-93c7-3b6447e6af3a admin admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:06:32.155402 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99c2d62-565f-40fe-93c7-3b6447e6af3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:06:32.155733 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99c2d62-565f-40fe-93c7-3b6447e6af3a admin admin] PATH_INFO: `/v3/domains` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:06:32.156440 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99c2d62-565f-40fe-93c7-3b6447e6af3a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81512) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:06:32.156804 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99c2d62-565f-40fe-93c7-3b6447e6af3a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:06:32.158511 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99c2d62-565f-40fe-93c7-3b6447e6af3a admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:06:32.175838 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1370/2765] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Aug 30 16:06:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:06:32.180623 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f348977-2f7f-4851-9fd9-90996b9ca1d0 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:06:32.191264 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f348977-2f7f-4851-9fd9-90996b9ca1d0 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:06:32.196426 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095e79ad-7c92-4465-98b8-18d990b4a677 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:06:32.197069 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095e79ad-7c92-4465-98b8-18d990b4a677 admin admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:06:32.197411 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095e79ad-7c92-4465-98b8-18d990b4a677 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:06:32.197708 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095e79ad-7c92-4465-98b8-18d990b4a677 admin admin] PATH_INFO: `/v3/projects` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:06:32.198356 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095e79ad-7c92-4465-98b8-18d990b4a677 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:06:32.201019 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095e79ad-7c92-4465-98b8-18d990b4a677 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:06:32.235923 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1396/2766] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Aug 30 16:06:32 2022] POST /identity/v3/projects => generated 389 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:06:32.240502 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99c2d62-565f-40fe-93c7-3b6447e6af3a admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:06:32.252268 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99c2d62-565f-40fe-93c7-3b6447e6af3a admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:06:32.323903 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b601a91-acd3-4ecc-8136-ce5bf391f459 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:06:32.324634 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b601a91-acd3-4ecc-8136-ce5bf391f459 admin admin] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:06:32.325117 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b601a91-acd3-4ecc-8136-ce5bf391f459 admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:06:32.325422 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b601a91-acd3-4ecc-8136-ce5bf391f459 admin admin] PATH_INFO: `/v3/users` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:06:32.326064 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b601a91-acd3-4ecc-8136-ce5bf391f459 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:06:32.327986 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b601a91-acd3-4ecc-8136-ce5bf391f459 admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:06:32.357737 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1371/2767] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Aug 30 16:06:32 2022] POST /identity/v3/users => generated 295 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:06:32.362018 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095e79ad-7c92-4465-98b8-18d990b4a677 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:06:32.377320 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095e79ad-7c92-4465-98b8-18d990b4a677 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:06:32.382452 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda324d9-c28d-4aaa-8761-37570e801bbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:06:32.383184 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda324d9-c28d-4aaa-8761-37570e801bbd admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:06:32.383647 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda324d9-c28d-4aaa-8761-37570e801bbd admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:06:32.384005 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda324d9-c28d-4aaa-8761-37570e801bbd admin admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:06:32.384896 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda324d9-c28d-4aaa-8761-37570e801bbd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:06:32.386746 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda324d9-c28d-4aaa-8761-37570e801bbd admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:06:32.398484 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1397/2768] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 30 16:06:32 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:06:32.402359 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b601a91-acd3-4ecc-8136-ce5bf391f459 admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:06:32.412884 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b601a91-acd3-4ecc-8136-ce5bf391f459 admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:06:32.419411 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e8df00-caa7-4a40-932a-fc7bf3240663 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:06:32.420120 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e8df00-caa7-4a40-932a-fc7bf3240663 admin admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:06:32.420754 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e8df00-caa7-4a40-932a-fc7bf3240663 admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:06:32.420754 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e8df00-caa7-4a40-932a-fc7bf3240663 admin admin] PATH_INFO: `/v3/roles` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:06:32.421595 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e8df00-caa7-4a40-932a-fc7bf3240663 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:06:32.423283 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e8df00-caa7-4a40-932a-fc7bf3240663 admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:06:32.434922 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1372/2769] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 30 16:06:32 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:06:32.439124 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda324d9-c28d-4aaa-8761-37570e801bbd admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:06:32.449698 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda324d9-c28d-4aaa-8761-37570e801bbd admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:06:32.455411 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6c98fd-5623-46e9-bd3b-10d26e90e214 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:06:32.456094 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6c98fd-5623-46e9-bd3b-10d26e90e214 admin admin] REQUEST_METHOD: `PUT` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:06:32.456501 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6c98fd-5623-46e9-bd3b-10d26e90e214 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:06:32.456784 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6c98fd-5623-46e9-bd3b-10d26e90e214 admin admin] PATH_INFO: `/v3/projects/27b5c784b1814011976f1a6f436895bf/users/d7007af897ae4b32807bf3b6b2ac3e06/roles/df8f9f844b1b439181d75a07c1a93418` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:06:32.461056 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:06:32.475993 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6c98fd-5623-46e9-bd3b-10d26e90e214 admin admin] RBAC: Authorizing `identity:create_grant(project_id=27b5c784b1814011976f1a6f436895bf, user_id=d7007af897ae4b32807bf3b6b2ac3e06, role_id=df8f9f844b1b439181d75a07c1a93418)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:06:32.479686 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6c98fd-5623-46e9-bd3b-10d26e90e214 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:06:32.498265 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1398/2770] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 30 16:06:32 2022] PUT /identity/v3/projects/27b5c784b1814011976f1a6f436895bf/users/d7007af897ae4b32807bf3b6b2ac3e06/roles/df8f9f844b1b439181d75a07c1a93418 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:06:32.510033 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abaa44a4-da27-4950-adfa-354429534646 None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:06:32.510319 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abaa44a4-da27-4950-adfa-354429534646 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:06:32.510585 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abaa44a4-da27-4950-adfa-354429534646 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:06:32.542899 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-abaa44a4-da27-4950-adfa-354429534646 None None] MFA Rules not processed for user `d7007af897ae4b32807bf3b6b2ac3e06`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:06:32.544658 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-abaa44a4-da27-4950-adfa-354429534646 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:06:32.560287 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:06:32.623866 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1373/2771] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 16:06:32 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 16:06:32.637140 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6c98fd-5623-46e9-bd3b-10d26e90e214 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:06:32.639110 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-0c6c98fd-5623-46e9-bd3b-10d26e90e214 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:06:32.706222 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6c98fd-5623-46e9-bd3b-10d26e90e214 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:06:32.710674 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f9199c-7d85-438e-9e79-0b4928e4dfa2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0100a04821804993866ea20f159663f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:06:32.711323 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f9199c-7d85-438e-9e79-0b4928e4dfa2 service cinder] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:06:32.711598 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f9199c-7d85-438e-9e79-0b4928e4dfa2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:06:32.711863 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f9199c-7d85-438e-9e79-0b4928e4dfa2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:06:32.722390 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f9199c-7d85-438e-9e79-0b4928e4dfa2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:06:32.723892 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f9199c-7d85-438e-9e79-0b4928e4dfa2 service cinder] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:06:32.734419 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1399/2772] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 16:06:32 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:06:42.623004 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=20 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:06:42.729641 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=32,keystone:INSERT=1 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:06:53.273961 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abaa44a4-da27-4950-adfa-354429534646 None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:06:53.340087 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abaa44a4-da27-4950-adfa-354429534646 None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:06:53.347543 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a708078-d646-4884-aca1-a8166b76ed59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:06:53.348169 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a708078-d646-4884-aca1-a8166b76ed59 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:06:53.348620 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a708078-d646-4884-aca1-a8166b76ed59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:06:53.348972 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a708078-d646-4884-aca1-a8166b76ed59 admin admin] PATH_INFO: `/v3/users/d7007af897ae4b32807bf3b6b2ac3e06` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:06:53.350345 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a708078-d646-4884-aca1-a8166b76ed59 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d7007af897ae4b32807bf3b6b2ac3e06)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:06:53.353615 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a708078-d646-4884-aca1-a8166b76ed59 admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:06:53.407398 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-1a708078-d646-4884-aca1-a8166b76ed59 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd7007af897ae4b32807bf3b6b2ac3e06'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:06:53.417408 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-1a708078-d646-4884-aca1-a8166b76ed59 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd7007af897ae4b32807bf3b6b2ac3e06'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:06:53.429795 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-1a708078-d646-4884-aca1-a8166b76ed59 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd7007af897ae4b32807bf3b6b2ac3e06'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:06:53.436977 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-1a708078-d646-4884-aca1-a8166b76ed59 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd7007af897ae4b32807bf3b6b2ac3e06'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:06:53.437684 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-1a708078-d646-4884-aca1-a8166b76ed59 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd7007af897ae4b32807bf3b6b2ac3e06'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:06:53.443688 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1374/2773] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Aug 30 16:06:53 2022] DELETE /identity/v3/users/d7007af897ae4b32807bf3b6b2ac3e06 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:06:53.448468 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84aac4ac-8297-4e02-a478-ba22a62b4c59 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:06:53.449011 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84aac4ac-8297-4e02-a478-ba22a62b4c59 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:06:53.449011 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84aac4ac-8297-4e02-a478-ba22a62b4c59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:06:53.468854 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-84aac4ac-8297-4e02-a478-ba22a62b4c59 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:06:53.470136 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-84aac4ac-8297-4e02-a478-ba22a62b4c59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:06:53.532508 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1400/2774] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 16:06:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:06:53.544132 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a708078-d646-4884-aca1-a8166b76ed59 admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:06:53.545733 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-1a708078-d646-4884-aca1-a8166b76ed59 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:06:53.607034 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a708078-d646-4884-aca1-a8166b76ed59 admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:06:53.610744 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5a3377-182d-427c-b489-49e9c57b3d2a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f4157105f3488f807b7475008ab722', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:06:53.611356 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5a3377-182d-427c-b489-49e9c57b3d2a service neutron] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:06:53.611633 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5a3377-182d-427c-b489-49e9c57b3d2a service neutron] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:06:53.611933 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5a3377-182d-427c-b489-49e9c57b3d2a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:06:53.621124 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5a3377-182d-427c-b489-49e9c57b3d2a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:06:53.622602 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5a3377-182d-427c-b489-49e9c57b3d2a service neutron] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:06:53.636167 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1375/2775] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 16:06:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:06:53.935932 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84aac4ac-8297-4e02-a478-ba22a62b4c59 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:06:53.937603 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-84aac4ac-8297-4e02-a478-ba22a62b4c59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:06:53.959470 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84aac4ac-8297-4e02-a478-ba22a62b4c59 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:06:53.964501 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7cac76-1166-4841-bed1-ede8a9ebf79f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:06:53.964938 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7cac76-1166-4841-bed1-ede8a9ebf79f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:06:53.965212 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7cac76-1166-4841-bed1-ede8a9ebf79f admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:06:53.965507 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7cac76-1166-4841-bed1-ede8a9ebf79f admin admin] PATH_INFO: `/v3/projects/27b5c784b1814011976f1a6f436895bf` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:06:53.966642 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7cac76-1166-4841-bed1-ede8a9ebf79f admin admin] RBAC: Authorizing `identity:delete_project(project_id=27b5c784b1814011976f1a6f436895bf)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:06:53.971382 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7cac76-1166-4841-bed1-ede8a9ebf79f admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:06:54.044978 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-cc7cac76-1166-4841-bed1-ede8a9ebf79f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '27b5c784b1814011976f1a6f436895bf', 'request_id': 'req-cc7cac76-1166-4841-bed1-ede8a9ebf79f'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:06:54.056728 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-cc7cac76-1166-4841-bed1-ede8a9ebf79f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '27b5c784b1814011976f1a6f436895bf', 'request_id': 'req-cc7cac76-1166-4841-bed1-ede8a9ebf79f'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:06:54.065491 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-cc7cac76-1166-4841-bed1-ede8a9ebf79f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '27b5c784b1814011976f1a6f436895bf', 'request_id': 'req-cc7cac76-1166-4841-bed1-ede8a9ebf79f'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:06:54.080739 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-cc7cac76-1166-4841-bed1-ede8a9ebf79f admin admin] The token cache is being invalidate because project 27b5c784b1814011976f1a6f436895bf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81511) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 16:06:54.081007 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-cc7cac76-1166-4841-bed1-ede8a9ebf79f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:06:54.081695 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-cc7cac76-1166-4841-bed1-ede8a9ebf79f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:06:54.083196 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1401/2776] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Aug 30 16:06:53 2022] DELETE /identity/v3/projects/27b5c784b1814011976f1a6f436895bf => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:07:03.232480 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d051d97-0657-4835-a99f-7227fe08473d None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:07:03.233017 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d051d97-0657-4835-a99f-7227fe08473d None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:07:03.233017 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d051d97-0657-4835-a99f-7227fe08473d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:07:03.253748 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-6d051d97-0657-4835-a99f-7227fe08473d None None] MFA Rules not processed for user `08661f6694ff44a0a4d15c792b54bb97`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:07:03.254935 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-6d051d97-0657-4835-a99f-7227fe08473d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:07:03.275378 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:07:03.317805 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1376/2777] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 16:07:03 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:07:03.554365 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7cac76-1166-4841-bed1-ede8a9ebf79f admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:07:03.556147 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-cc7cac76-1166-4841-bed1-ede8a9ebf79f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:07:03.616889 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7cac76-1166-4841-bed1-ede8a9ebf79f admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:07:03.621202 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b082c1d-ef56-4a8d-828d-d6bf7979409d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f4157105f3488f807b7475008ab722', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:07:03.621843 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b082c1d-ef56-4a8d-828d-d6bf7979409d service neutron] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:07:03.622120 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b082c1d-ef56-4a8d-828d-d6bf7979409d service neutron] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:07:03.622393 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b082c1d-ef56-4a8d-828d-d6bf7979409d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:07:03.624396 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-7b082c1d-ef56-4a8d-828d-d6bf7979409d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:07:03.634877 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b082c1d-ef56-4a8d-828d-d6bf7979409d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:07:03.636314 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b082c1d-ef56-4a8d-828d-d6bf7979409d service neutron] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:07:03.644674 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1402/2778] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 16:07:03 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:07:13.310545 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=15 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:07:13.641145 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=80,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:07:26.001013 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d051d97-0657-4835-a99f-7227fe08473d None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:07:26.003792 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-6d051d97-0657-4835-a99f-7227fe08473d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:07:26.049808 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d051d97-0657-4835-a99f-7227fe08473d None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:07:26.055074 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d9c5b5-d21b-4dd8-b371-74cb8db70b53 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f4157105f3488f807b7475008ab722', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:07:26.056053 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d9c5b5-d21b-4dd8-b371-74cb8db70b53 service neutron] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:07:26.056516 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d9c5b5-d21b-4dd8-b371-74cb8db70b53 service neutron] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:07:26.057915 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d9c5b5-d21b-4dd8-b371-74cb8db70b53 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:07:26.059262 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-05d9c5b5-d21b-4dd8-b371-74cb8db70b53 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:07:26.143977 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d9c5b5-d21b-4dd8-b371-74cb8db70b53 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:07:26.145916 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d9c5b5-d21b-4dd8-b371-74cb8db70b53 service neutron] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:07:26.155195 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1377/2779] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 16:07:25 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:07:31.131140 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b082c1d-ef56-4a8d-828d-d6bf7979409d service neutron] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:07:31.133714 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-7b082c1d-ef56-4a8d-828d-d6bf7979409d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:07:31.289056 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b082c1d-ef56-4a8d-828d-d6bf7979409d service neutron] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:07:31.294123 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8c4389-07b9-466b-b405-34ebd0792f3d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc12c6d31eab4a17a160250adc2380c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:07:31.294727 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8c4389-07b9-466b-b405-34ebd0792f3d service placement] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:07:31.294999 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8c4389-07b9-466b-b405-34ebd0792f3d service placement] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:07:31.295263 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8c4389-07b9-466b-b405-34ebd0792f3d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:07:31.297004 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-ae8c4389-07b9-466b-b405-34ebd0792f3d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:07:31.308592 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8c4389-07b9-466b-b405-34ebd0792f3d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:07:31.310197 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8c4389-07b9-466b-b405-34ebd0792f3d service placement] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:07:31.321048 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1403/2780] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 16:07:31 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:07:34.723672 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d9c5b5-d21b-4dd8-b371-74cb8db70b53 service neutron] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:07:34.746195 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d9c5b5-d21b-4dd8-b371-74cb8db70b53 service neutron] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:07:34.750286 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0948353f-7c93-4af8-a223-b534ff11b2a4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f4157105f3488f807b7475008ab722', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:07:34.751069 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0948353f-7c93-4af8-a223-b534ff11b2a4 service neutron] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:07:34.751561 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0948353f-7c93-4af8-a223-b534ff11b2a4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:07:34.751828 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0948353f-7c93-4af8-a223-b534ff11b2a4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:07:34.753876 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-0948353f-7c93-4af8-a223-b534ff11b2a4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:07:34.845840 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0948353f-7c93-4af8-a223-b534ff11b2a4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:07:34.847316 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0948353f-7c93-4af8-a223-b534ff11b2a4 service neutron] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:07:34.866934 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1378/2781] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 16:07:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:07:41.316235 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=57 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:07:41.434494 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8c4389-07b9-466b-b405-34ebd0792f3d service placement] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:07:41.435933 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-ae8c4389-07b9-466b-b405-34ebd0792f3d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:07:41.451321 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8c4389-07b9-466b-b405-34ebd0792f3d service placement] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:07:41.456118 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005734ce-5946-4635-9a14-a584fdd37a99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:07:41.456764 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005734ce-5946-4635-9a14-a584fdd37a99 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:07:41.457042 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005734ce-5946-4635-9a14-a584fdd37a99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:07:41.457355 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005734ce-5946-4635-9a14-a584fdd37a99 admin admin] PATH_INFO: `/v3/users/4f00011535aa4836b1cc410dd5d97beb` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:07:41.458505 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005734ce-5946-4635-9a14-a584fdd37a99 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4f00011535aa4836b1cc410dd5d97beb)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:07:41.460516 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005734ce-5946-4635-9a14-a584fdd37a99 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:07:41.516116 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-005734ce-5946-4635-9a14-a584fdd37a99 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4f00011535aa4836b1cc410dd5d97beb'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:07:41.528725 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-005734ce-5946-4635-9a14-a584fdd37a99 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4f00011535aa4836b1cc410dd5d97beb'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:07:41.537888 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-005734ce-5946-4635-9a14-a584fdd37a99 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4f00011535aa4836b1cc410dd5d97beb'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:07:41.538605 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-005734ce-5946-4635-9a14-a584fdd37a99 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4f00011535aa4836b1cc410dd5d97beb'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:07:41.546045 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-005734ce-5946-4635-9a14-a584fdd37a99 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4f00011535aa4836b1cc410dd5d97beb'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:07:41.551967 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1404/2782] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Aug 30 16:07:41 2022] DELETE /identity/v3/users/4f00011535aa4836b1cc410dd5d97beb => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:07:41.738236 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0948353f-7c93-4af8-a223-b534ff11b2a4 service neutron] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:07:41.749378 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0948353f-7c93-4af8-a223-b534ff11b2a4 service neutron] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:07:41.759317 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=100 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:07:41.808986 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce92c6e-8116-4688-925d-74246c4fe854 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:07:41.809648 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce92c6e-8116-4688-925d-74246c4fe854 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:07:41.809925 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce92c6e-8116-4688-925d-74246c4fe854 admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:07:41.810196 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce92c6e-8116-4688-925d-74246c4fe854 admin admin] PATH_INFO: `/v3/projects/340ea9b1270443f58c051cfb5699caa5` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:07:41.811317 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce92c6e-8116-4688-925d-74246c4fe854 admin admin] RBAC: Authorizing `identity:delete_project(project_id=340ea9b1270443f58c051cfb5699caa5)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:07:41.813251 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce92c6e-8116-4688-925d-74246c4fe854 admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:07:41.863641 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-8ce92c6e-8116-4688-925d-74246c4fe854 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '340ea9b1270443f58c051cfb5699caa5', 'request_id': 'req-8ce92c6e-8116-4688-925d-74246c4fe854'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:07:41.872660 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-8ce92c6e-8116-4688-925d-74246c4fe854 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '340ea9b1270443f58c051cfb5699caa5', 'request_id': 'req-8ce92c6e-8116-4688-925d-74246c4fe854'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:07:41.878826 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-8ce92c6e-8116-4688-925d-74246c4fe854 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '340ea9b1270443f58c051cfb5699caa5', 'request_id': 'req-8ce92c6e-8116-4688-925d-74246c4fe854'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:07:41.889283 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-8ce92c6e-8116-4688-925d-74246c4fe854 admin admin] The token cache is being invalidate because project 340ea9b1270443f58c051cfb5699caa5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81512) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 16:07:41.889573 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-8ce92c6e-8116-4688-925d-74246c4fe854 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:07:41.890206 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-8ce92c6e-8116-4688-925d-74246c4fe854 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:07:41.891715 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1379/2783] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Aug 30 16:07:41 2022] DELETE /identity/v3/projects/340ea9b1270443f58c051cfb5699caa5 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:07:41.895853 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005734ce-5946-4635-9a14-a584fdd37a99 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:07:41.897484 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-005734ce-5946-4635-9a14-a584fdd37a99 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:07:41.911232 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005734ce-5946-4635-9a14-a584fdd37a99 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:07:41.975184 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4277f7-5729-412a-a7e8-56431fa8bb64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:07:41.975857 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4277f7-5729-412a-a7e8-56431fa8bb64 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:07:41.976134 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4277f7-5729-412a-a7e8-56431fa8bb64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:07:41.976404 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4277f7-5729-412a-a7e8-56431fa8bb64 admin admin] PATH_INFO: `/v3/users/43dc309356e14c319e92f55568e93afd` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:07:41.977973 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4277f7-5729-412a-a7e8-56431fa8bb64 admin admin] RBAC: Authorizing `identity:delete_user(user_id=43dc309356e14c319e92f55568e93afd)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:07:41.980414 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4277f7-5729-412a-a7e8-56431fa8bb64 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:07:42.034285 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-be4277f7-5729-412a-a7e8-56431fa8bb64 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '43dc309356e14c319e92f55568e93afd'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:07:42.049506 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-be4277f7-5729-412a-a7e8-56431fa8bb64 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '43dc309356e14c319e92f55568e93afd'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:07:42.058586 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-be4277f7-5729-412a-a7e8-56431fa8bb64 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '43dc309356e14c319e92f55568e93afd'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:07:42.059231 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-be4277f7-5729-412a-a7e8-56431fa8bb64 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '43dc309356e14c319e92f55568e93afd'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:07:42.061414 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:07:42.068478 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-be4277f7-5729-412a-a7e8-56431fa8bb64 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '43dc309356e14c319e92f55568e93afd'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:07:42.075650 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1405/2784] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Aug 30 16:07:41 2022] DELETE /identity/v3/users/43dc309356e14c319e92f55568e93afd => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:07:42.256211 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce92c6e-8116-4688-925d-74246c4fe854 admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:07:42.257671 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-8ce92c6e-8116-4688-925d-74246c4fe854 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:07:42.283297 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce92c6e-8116-4688-925d-74246c4fe854 admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:07:42.348530 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43af17bf-94fb-4407-897d-4ec0aa2bc59f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:07:42.349173 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43af17bf-94fb-4407-897d-4ec0aa2bc59f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:07:42.349585 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43af17bf-94fb-4407-897d-4ec0aa2bc59f admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:07:42.349884 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43af17bf-94fb-4407-897d-4ec0aa2bc59f admin admin] PATH_INFO: `/v3/projects/4ab1c607d41344b18931ea65951813dc` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:07:42.351190 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43af17bf-94fb-4407-897d-4ec0aa2bc59f admin admin] RBAC: Authorizing `identity:delete_project(project_id=4ab1c607d41344b18931ea65951813dc)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:07:42.353170 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43af17bf-94fb-4407-897d-4ec0aa2bc59f admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:07:42.416841 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:07:42.420195 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-43af17bf-94fb-4407-897d-4ec0aa2bc59f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4ab1c607d41344b18931ea65951813dc', 'request_id': 'req-43af17bf-94fb-4407-897d-4ec0aa2bc59f'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:07:42.435718 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-43af17bf-94fb-4407-897d-4ec0aa2bc59f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4ab1c607d41344b18931ea65951813dc', 'request_id': 'req-43af17bf-94fb-4407-897d-4ec0aa2bc59f'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:07:42.443758 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-43af17bf-94fb-4407-897d-4ec0aa2bc59f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4ab1c607d41344b18931ea65951813dc', 'request_id': 'req-43af17bf-94fb-4407-897d-4ec0aa2bc59f'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:07:42.459559 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-43af17bf-94fb-4407-897d-4ec0aa2bc59f admin admin] The token cache is being invalidate because project 4ab1c607d41344b18931ea65951813dc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81512) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 16:07:42.459853 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-43af17bf-94fb-4407-897d-4ec0aa2bc59f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:07:42.460758 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-43af17bf-94fb-4407-897d-4ec0aa2bc59f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:07:42.462940 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1380/2785] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Aug 30 16:07:42 2022] DELETE /identity/v3/projects/4ab1c607d41344b18931ea65951813dc => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:07:52.073025 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=4 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:07:52.454417 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:DELETE=4,keystone:SELECT=7,keystone:INSERT=1 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:08:08.077881 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4277f7-5729-412a-a7e8-56431fa8bb64 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:08:08.079438 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-be4277f7-5729-412a-a7e8-56431fa8bb64 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:08:08.093906 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4277f7-5729-412a-a7e8-56431fa8bb64 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:08:08.151158 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062b9a78-53ae-4827-a1c4-d1b697d9749a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:08:08.151814 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062b9a78-53ae-4827-a1c4-d1b697d9749a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:08:08.152088 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062b9a78-53ae-4827-a1c4-d1b697d9749a admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:08:08.152356 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062b9a78-53ae-4827-a1c4-d1b697d9749a admin admin] PATH_INFO: `/v3/users/08661f6694ff44a0a4d15c792b54bb97` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:08:08.153713 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062b9a78-53ae-4827-a1c4-d1b697d9749a admin admin] RBAC: Authorizing `identity:delete_user(user_id=08661f6694ff44a0a4d15c792b54bb97)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:08:08.155647 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062b9a78-53ae-4827-a1c4-d1b697d9749a admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:08:08.213978 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-062b9a78-53ae-4827-a1c4-d1b697d9749a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '08661f6694ff44a0a4d15c792b54bb97'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:08:08.226302 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-062b9a78-53ae-4827-a1c4-d1b697d9749a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '08661f6694ff44a0a4d15c792b54bb97'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:08:08.236716 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-062b9a78-53ae-4827-a1c4-d1b697d9749a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '08661f6694ff44a0a4d15c792b54bb97'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:08:08.237465 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-062b9a78-53ae-4827-a1c4-d1b697d9749a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '08661f6694ff44a0a4d15c792b54bb97'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:08:08.245009 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-062b9a78-53ae-4827-a1c4-d1b697d9749a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '08661f6694ff44a0a4d15c792b54bb97'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:08:08.251246 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1406/2786] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Aug 30 16:08:08 2022] DELETE /identity/v3/users/08661f6694ff44a0a4d15c792b54bb97 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:08:08.442493 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43af17bf-94fb-4407-897d-4ec0aa2bc59f admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:08:08.444067 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-43af17bf-94fb-4407-897d-4ec0aa2bc59f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:08:08.461123 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43af17bf-94fb-4407-897d-4ec0aa2bc59f admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:08:08.529950 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775db0b2-0af6-425d-bb3a-a65f7120f4f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:08:08.530715 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775db0b2-0af6-425d-bb3a-a65f7120f4f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:08:08.530988 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775db0b2-0af6-425d-bb3a-a65f7120f4f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:08:08.531307 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775db0b2-0af6-425d-bb3a-a65f7120f4f3 admin admin] PATH_INFO: `/v3/projects/ebf5f0daa032440ab81e5abda55f6e82` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:08:08.533262 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775db0b2-0af6-425d-bb3a-a65f7120f4f3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ebf5f0daa032440ab81e5abda55f6e82)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:08:08.535915 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775db0b2-0af6-425d-bb3a-a65f7120f4f3 admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:08:08.585598 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-775db0b2-0af6-425d-bb3a-a65f7120f4f3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ebf5f0daa032440ab81e5abda55f6e82', 'request_id': 'req-775db0b2-0af6-425d-bb3a-a65f7120f4f3'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:08:08.593782 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-775db0b2-0af6-425d-bb3a-a65f7120f4f3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ebf5f0daa032440ab81e5abda55f6e82', 'request_id': 'req-775db0b2-0af6-425d-bb3a-a65f7120f4f3'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:08:08.600296 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-775db0b2-0af6-425d-bb3a-a65f7120f4f3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ebf5f0daa032440ab81e5abda55f6e82', 'request_id': 'req-775db0b2-0af6-425d-bb3a-a65f7120f4f3'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:08:08.611040 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-775db0b2-0af6-425d-bb3a-a65f7120f4f3 admin admin] The token cache is being invalidate because project ebf5f0daa032440ab81e5abda55f6e82 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81512) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 16:08:08.611324 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-775db0b2-0af6-425d-bb3a-a65f7120f4f3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:08:08.611940 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-775db0b2-0af6-425d-bb3a-a65f7120f4f3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:08:08.613462 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1381/2787] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Aug 30 16:08:08 2022] DELETE /identity/v3/projects/ebf5f0daa032440ab81e5abda55f6e82 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:08:08.618453 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2b624c-4baa-490b-b476-06fd14c6d158 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:08:08.618791 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2b624c-4baa-490b-b476-06fd14c6d158 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:08:08.619062 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2b624c-4baa-490b-b476-06fd14c6d158 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:08:08.678774 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-8c2b624c-4baa-490b-b476-06fd14c6d158 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:08:08.680120 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-8c2b624c-4baa-490b-b476-06fd14c6d158 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:08:08.725914 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:08:08.748672 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1407/2788] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 16:08:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:08:08.759628 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c60b717-0681-42f7-b7ec-9d01e3c0534b None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:08:08.759912 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c60b717-0681-42f7-b7ec-9d01e3c0534b None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:08:08.760181 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c60b717-0681-42f7-b7ec-9d01e3c0534b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:08:08.780328 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-5c60b717-0681-42f7-b7ec-9d01e3c0534b None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:08:08.781607 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-5c60b717-0681-42f7-b7ec-9d01e3c0534b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:08:08.790086 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1382/2789] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 16:08:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:08:08.795154 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2b624c-4baa-490b-b476-06fd14c6d158 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:08:08.810691 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2b624c-4baa-490b-b476-06fd14c6d158 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:08:08.820307 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c1d95a-4fab-4f90-b73c-906a9c052fd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:08:08.820958 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c1d95a-4fab-4f90-b73c-906a9c052fd5 admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:08:08.821256 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c1d95a-4fab-4f90-b73c-906a9c052fd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:08:08.821534 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c1d95a-4fab-4f90-b73c-906a9c052fd5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:08:08.822184 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c1d95a-4fab-4f90-b73c-906a9c052fd5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:08:08.822486 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c1d95a-4fab-4f90-b73c-906a9c052fd5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:08:08.824343 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c1d95a-4fab-4f90-b73c-906a9c052fd5 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:08:08.839554 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1408/2790] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Aug 30 16:08:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:08:08.843980 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c60b717-0681-42f7-b7ec-9d01e3c0534b None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:08:08.853878 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c60b717-0681-42f7-b7ec-9d01e3c0534b None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:08:08.859619 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4118622-e5d2-486e-ba69-6e518494dc93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:08:08.860214 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4118622-e5d2-486e-ba69-6e518494dc93 admin admin] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:08:08.860489 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4118622-e5d2-486e-ba69-6e518494dc93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:08:08.860753 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4118622-e5d2-486e-ba69-6e518494dc93 admin admin] PATH_INFO: `/v3/projects` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:08:08.861423 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4118622-e5d2-486e-ba69-6e518494dc93 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:08:08.863341 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4118622-e5d2-486e-ba69-6e518494dc93 admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:08:08.893379 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1383/2791] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Aug 30 16:08:08 2022] POST /identity/v3/projects => generated 375 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:08:08.899254 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c1d95a-4fab-4f90-b73c-906a9c052fd5 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:08:08.910314 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c1d95a-4fab-4f90-b73c-906a9c052fd5 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:08:08.971962 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd1a3e2-d5db-42f5-8a37-e7ce93cbc0b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:08:08.972558 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd1a3e2-d5db-42f5-8a37-e7ce93cbc0b7 admin admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:08:08.972846 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd1a3e2-d5db-42f5-8a37-e7ce93cbc0b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:08:08.973115 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd1a3e2-d5db-42f5-8a37-e7ce93cbc0b7 admin admin] PATH_INFO: `/v3/users` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:08:08.973783 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd1a3e2-d5db-42f5-8a37-e7ce93cbc0b7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:08:08.975701 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd1a3e2-d5db-42f5-8a37-e7ce93cbc0b7 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:08:08.998147 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1409/2792] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Aug 30 16:08:08 2022] POST /identity/v3/users => generated 288 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:08:09.002591 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4118622-e5d2-486e-ba69-6e518494dc93 admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:08:09.013353 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4118622-e5d2-486e-ba69-6e518494dc93 admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:08:09.018166 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9eafe3-1f03-41e2-863e-045f402cf9a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:08:09.018758 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9eafe3-1f03-41e2-863e-045f402cf9a7 admin admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:08:09.019034 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9eafe3-1f03-41e2-863e-045f402cf9a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:08:09.019298 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9eafe3-1f03-41e2-863e-045f402cf9a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:08:09.019934 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9eafe3-1f03-41e2-863e-045f402cf9a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:08:09.021566 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9eafe3-1f03-41e2-863e-045f402cf9a7 admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:08:09.031675 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1384/2793] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 30 16:08:09 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:08:09.035758 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd1a3e2-d5db-42f5-8a37-e7ce93cbc0b7 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:08:09.046331 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd1a3e2-d5db-42f5-8a37-e7ce93cbc0b7 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:08:09.051387 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28da15a2-cca2-4139-83d6-7fb6aad7aed3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:08:09.051979 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28da15a2-cca2-4139-83d6-7fb6aad7aed3 admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:08:09.052253 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28da15a2-cca2-4139-83d6-7fb6aad7aed3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:08:09.052519 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28da15a2-cca2-4139-83d6-7fb6aad7aed3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:08:09.053163 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28da15a2-cca2-4139-83d6-7fb6aad7aed3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:08:09.054815 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28da15a2-cca2-4139-83d6-7fb6aad7aed3 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:08:09.065525 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1410/2794] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 30 16:08:09 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:08:09.069561 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9eafe3-1f03-41e2-863e-045f402cf9a7 admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:08:09.079402 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9eafe3-1f03-41e2-863e-045f402cf9a7 admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:08:09.084246 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c9c8f3-836a-4259-a7fb-6e5942b9d9a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:08:09.084953 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c9c8f3-836a-4259-a7fb-6e5942b9d9a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:08:09.085509 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c9c8f3-836a-4259-a7fb-6e5942b9d9a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:08:09.085628 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c9c8f3-836a-4259-a7fb-6e5942b9d9a6 admin admin] PATH_INFO: `/v3/projects/330e7c1cf5894ac9b6d21915c5f645bc/users/02287f232cdd4842a747bb749b483cc3/roles/df8f9f844b1b439181d75a07c1a93418` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:08:09.100835 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c9c8f3-836a-4259-a7fb-6e5942b9d9a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=330e7c1cf5894ac9b6d21915c5f645bc, user_id=02287f232cdd4842a747bb749b483cc3, role_id=df8f9f844b1b439181d75a07c1a93418)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:08:09.103837 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c9c8f3-836a-4259-a7fb-6e5942b9d9a6 admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:08:09.119345 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1385/2795] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 30 16:08:09 2022] PUT /identity/v3/projects/330e7c1cf5894ac9b6d21915c5f645bc/users/02287f232cdd4842a747bb749b483cc3/roles/df8f9f844b1b439181d75a07c1a93418 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:08:09.124485 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843eddb3-b84c-4a8c-8419-f717f2c5efde None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:08:09.124975 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843eddb3-b84c-4a8c-8419-f717f2c5efde None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:08:09.125082 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843eddb3-b84c-4a8c-8419-f717f2c5efde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:08:09.144837 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-843eddb3-b84c-4a8c-8419-f717f2c5efde None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:08:09.146066 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-843eddb3-b84c-4a8c-8419-f717f2c5efde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:08:09.213353 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1411/2796] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 16:08:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 16:08:09.223407 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c9c8f3-836a-4259-a7fb-6e5942b9d9a6 admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:08:09.225518 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-60c9c8f3-836a-4259-a7fb-6e5942b9d9a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:08:09.257994 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:08:09.299512 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c9c8f3-836a-4259-a7fb-6e5942b9d9a6 admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:08:09.303329 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dea1a6-5c15-4f4e-8f64-a8a8c0727ed9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f4157105f3488f807b7475008ab722', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:08:09.303988 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dea1a6-5c15-4f4e-8f64-a8a8c0727ed9 service neutron] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:08:09.304263 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dea1a6-5c15-4f4e-8f64-a8a8c0727ed9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:08:09.304568 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dea1a6-5c15-4f4e-8f64-a8a8c0727ed9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:08:09.313585 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7dea1a6-5c15-4f4e-8f64-a8a8c0727ed9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:08:09.315049 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7dea1a6-5c15-4f4e-8f64-a8a8c0727ed9 service neutron] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:08:09.324389 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1386/2797] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 16:08:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:08:13.264019 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a56b43-298e-41c0-9350-3eca5d28d42e None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:08:13.264925 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a56b43-298e-41c0-9350-3eca5d28d42e None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:08:13.265407 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a56b43-298e-41c0-9350-3eca5d28d42e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:08:13.302033 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-c1a56b43-298e-41c0-9350-3eca5d28d42e None None] MFA Rules not processed for user `02287f232cdd4842a747bb749b483cc3`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:08:13.303622 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-c1a56b43-298e-41c0-9350-3eca5d28d42e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:08:13.353513 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:08:13.375384 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1412/2798] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 16:08:13 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:08:13.380090 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dea1a6-5c15-4f4e-8f64-a8a8c0727ed9 service neutron] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:08:13.390242 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dea1a6-5c15-4f4e-8f64-a8a8c0727ed9 service neutron] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:08:13.395103 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db7f71e-fdf0-4b27-a105-c8de9fc38387 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:08:13.395707 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db7f71e-fdf0-4b27-a105-c8de9fc38387 admin admin] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:08:13.395985 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db7f71e-fdf0-4b27-a105-c8de9fc38387 admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:08:13.396236 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db7f71e-fdf0-4b27-a105-c8de9fc38387 admin admin] PATH_INFO: `/v3/projects` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:08:13.396876 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db7f71e-fdf0-4b27-a105-c8de9fc38387 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:08:13.398948 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db7f71e-fdf0-4b27-a105-c8de9fc38387 admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:08:13.428613 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1387/2799] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Aug 30 16:08:13 2022] POST /identity/v3/projects => generated 377 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 16:08:13.432675 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a56b43-298e-41c0-9350-3eca5d28d42e None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:08:13.444773 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a56b43-298e-41c0-9350-3eca5d28d42e None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:08:13.524701 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1666583a-7f55-4ff5-9b8a-99e3baa6ea73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:08:13.525357 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1666583a-7f55-4ff5-9b8a-99e3baa6ea73 admin admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:08:13.525631 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1666583a-7f55-4ff5-9b8a-99e3baa6ea73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:08:13.525898 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1666583a-7f55-4ff5-9b8a-99e3baa6ea73 admin admin] PATH_INFO: `/v3/users` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:08:13.526551 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1666583a-7f55-4ff5-9b8a-99e3baa6ea73 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:08:13.528477 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1666583a-7f55-4ff5-9b8a-99e3baa6ea73 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:08:13.553503 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1413/2800] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Aug 30 16:08:13 2022] POST /identity/v3/users => generated 295 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:08:13.557665 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db7f71e-fdf0-4b27-a105-c8de9fc38387 admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:08:13.569588 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db7f71e-fdf0-4b27-a105-c8de9fc38387 admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:08:13.575186 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26acc864-0a8d-4ffe-a00a-849f9ecd021b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:08:13.575800 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26acc864-0a8d-4ffe-a00a-849f9ecd021b admin admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:08:13.576073 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26acc864-0a8d-4ffe-a00a-849f9ecd021b admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:08:13.576344 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26acc864-0a8d-4ffe-a00a-849f9ecd021b admin admin] PATH_INFO: `/v3/roles` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:08:13.576989 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26acc864-0a8d-4ffe-a00a-849f9ecd021b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:08:13.578774 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26acc864-0a8d-4ffe-a00a-849f9ecd021b admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:08:13.589766 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1388/2801] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 30 16:08:13 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:08:13.594151 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1666583a-7f55-4ff5-9b8a-99e3baa6ea73 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:08:13.604916 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1666583a-7f55-4ff5-9b8a-99e3baa6ea73 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:08:13.609681 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cb4ef3-964a-49cb-8311-8a10a5a0c100 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:08:13.610362 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cb4ef3-964a-49cb-8311-8a10a5a0c100 admin admin] REQUEST_METHOD: `PUT` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:08:13.610644 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cb4ef3-964a-49cb-8311-8a10a5a0c100 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:08:13.610914 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cb4ef3-964a-49cb-8311-8a10a5a0c100 admin admin] PATH_INFO: `/v3/projects/1231f24ce4ae4692b2fde5e2a63debef/users/579b15c74c00457184dcd978b929dbbb/roles/f265386950a641ac8413b53eda4ae34d` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:08:13.630305 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cb4ef3-964a-49cb-8311-8a10a5a0c100 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1231f24ce4ae4692b2fde5e2a63debef, user_id=579b15c74c00457184dcd978b929dbbb, role_id=f265386950a641ac8413b53eda4ae34d)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:08:13.633073 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cb4ef3-964a-49cb-8311-8a10a5a0c100 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:08:13.651136 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1414/2802] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 30 16:08:13 2022] PUT /identity/v3/projects/1231f24ce4ae4692b2fde5e2a63debef/users/579b15c74c00457184dcd978b929dbbb/roles/f265386950a641ac8413b53eda4ae34d => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:08:13.655203 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26acc864-0a8d-4ffe-a00a-849f9ecd021b admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:08:13.665421 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26acc864-0a8d-4ffe-a00a-849f9ecd021b admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:08:13.727118 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37efe5ea-dc72-4bd4-9d7a-af9a597748db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:08:13.728077 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37efe5ea-dc72-4bd4-9d7a-af9a597748db admin admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:08:13.728350 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37efe5ea-dc72-4bd4-9d7a-af9a597748db admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:08:13.728616 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37efe5ea-dc72-4bd4-9d7a-af9a597748db admin admin] PATH_INFO: `/v3/roles` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:08:13.729290 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37efe5ea-dc72-4bd4-9d7a-af9a597748db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:08:13.730894 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37efe5ea-dc72-4bd4-9d7a-af9a597748db admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:08:13.741914 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1389/2803] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 30 16:08:13 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:08:13.746320 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cb4ef3-964a-49cb-8311-8a10a5a0c100 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:08:13.757316 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cb4ef3-964a-49cb-8311-8a10a5a0c100 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:08:13.762158 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606ebd9d-19cf-421b-bade-ecf8a17a868a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:08:13.763044 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606ebd9d-19cf-421b-bade-ecf8a17a868a admin admin] REQUEST_METHOD: `PUT` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:08:13.763483 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606ebd9d-19cf-421b-bade-ecf8a17a868a admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:08:13.763883 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606ebd9d-19cf-421b-bade-ecf8a17a868a admin admin] PATH_INFO: `/v3/projects/1231f24ce4ae4692b2fde5e2a63debef/users/579b15c74c00457184dcd978b929dbbb/roles/f265386950a641ac8413b53eda4ae34d` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:08:13.766137 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606ebd9d-19cf-421b-bade-ecf8a17a868a admin admin] RBAC: Authorizing `identity:create_grant(project_id=1231f24ce4ae4692b2fde5e2a63debef, user_id=579b15c74c00457184dcd978b929dbbb, role_id=f265386950a641ac8413b53eda4ae34d)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:08:13.768767 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606ebd9d-19cf-421b-bade-ecf8a17a868a admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:08:13.779919 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1415/2804] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 30 16:08:13 2022] PUT /identity/v3/projects/1231f24ce4ae4692b2fde5e2a63debef/users/579b15c74c00457184dcd978b929dbbb/roles/f265386950a641ac8413b53eda4ae34d => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 16:08:18.234228 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8cf4f8-5af5-47f9-9e5a-c62892151cc7 None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:08:18.235223 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8cf4f8-5af5-47f9-9e5a-c62892151cc7 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:08:18.235561 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8cf4f8-5af5-47f9-9e5a-c62892151cc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:08:18.274487 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-9d8cf4f8-5af5-47f9-9e5a-c62892151cc7 None None] MFA Rules not processed for user `579b15c74c00457184dcd978b929dbbb`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:08:18.276325 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-9d8cf4f8-5af5-47f9-9e5a-c62892151cc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:08:18.350582 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1390/2805] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 16:08:18 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:08:18.367486 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606ebd9d-19cf-421b-bade-ecf8a17a868a admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:08:18.370525 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-606ebd9d-19cf-421b-bade-ecf8a17a868a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:08:18.528192 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606ebd9d-19cf-421b-bade-ecf8a17a868a admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:08:18.534280 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bddfa5-4952-4c04-b959-a292f5e7013e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a1e756ae9be4a7bac1e81acf07e8f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:08:18.535010 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bddfa5-4952-4c04-b959-a292f5e7013e service nova] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:08:18.535355 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bddfa5-4952-4c04-b959-a292f5e7013e service nova] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:08:18.535691 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bddfa5-4952-4c04-b959-a292f5e7013e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:08:18.541490 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:08:18.600450 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bddfa5-4952-4c04-b959-a292f5e7013e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:08:18.603951 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bddfa5-4952-4c04-b959-a292f5e7013e service nova] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:08:18.623199 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1416/2806] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 16:08:18 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 256 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:08:28.347138 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=96,keystone:INSERT=1 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:08:28.617069 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=24 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:08:33.965357 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511ef714-3fda-4efd-aa6c-ee4a311aa275 None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:08:33.965925 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511ef714-3fda-4efd-aa6c-ee4a311aa275 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:08:33.965925 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511ef714-3fda-4efd-aa6c-ee4a311aa275 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:08:33.986633 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-511ef714-3fda-4efd-aa6c-ee4a311aa275 None None] MFA Rules not processed for user `02287f232cdd4842a747bb749b483cc3`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:08:33.987842 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-511ef714-3fda-4efd-aa6c-ee4a311aa275 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:08:33.998290 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1391/2807] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 16:08:33 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:08:43.980509 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=3 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:08:52.032478 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bddfa5-4952-4c04-b959-a292f5e7013e service nova] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:08:52.034178 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-c5bddfa5-4952-4c04-b959-a292f5e7013e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:08:52.095058 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bddfa5-4952-4c04-b959-a292f5e7013e service nova] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:08:52.099491 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aed941c-0fd9-4f55-91a1-aecd4f476edf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f4157105f3488f807b7475008ab722', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:08:52.100097 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aed941c-0fd9-4f55-91a1-aecd4f476edf service neutron] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:08:52.100372 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aed941c-0fd9-4f55-91a1-aecd4f476edf service neutron] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:08:52.100669 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aed941c-0fd9-4f55-91a1-aecd4f476edf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:08:52.112575 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aed941c-0fd9-4f55-91a1-aecd4f476edf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:08:52.114872 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aed941c-0fd9-4f55-91a1-aecd4f476edf service neutron] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:08:52.130063 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1417/2808] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 16:08:52 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:09:02.123690 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=26 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:09:30.333795 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511ef714-3fda-4efd-aa6c-ee4a311aa275 None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:09:30.348494 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511ef714-3fda-4efd-aa6c-ee4a311aa275 None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:09:30.355473 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cbdaf0-4eac-44df-8ee4-eb3c1b72cd88 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f4157105f3488f807b7475008ab722', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:09:30.356221 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cbdaf0-4eac-44df-8ee4-eb3c1b72cd88 service neutron] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:09:30.356542 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cbdaf0-4eac-44df-8ee4-eb3c1b72cd88 service neutron] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:09:30.356890 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cbdaf0-4eac-44df-8ee4-eb3c1b72cd88 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:09:30.360145 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-d0cbdaf0-4eac-44df-8ee4-eb3c1b72cd88 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:09:30.376205 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0cbdaf0-4eac-44df-8ee4-eb3c1b72cd88 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:09:30.377714 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0cbdaf0-4eac-44df-8ee4-eb3c1b72cd88 service neutron] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:09:30.388703 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1392/2809] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 16:09:30 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 16:09:31.581096 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aed941c-0fd9-4f55-91a1-aecd4f476edf service neutron] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:09:31.590146 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aed941c-0fd9-4f55-91a1-aecd4f476edf service neutron] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:09:31.594436 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fca8195-a678-4212-83b7-1a230a4f2039 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f4157105f3488f807b7475008ab722', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:09:31.595031 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fca8195-a678-4212-83b7-1a230a4f2039 service neutron] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:09:31.595305 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fca8195-a678-4212-83b7-1a230a4f2039 service neutron] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:09:31.595603 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fca8195-a678-4212-83b7-1a230a4f2039 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:09:31.597583 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-8fca8195-a678-4212-83b7-1a230a4f2039 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:09:31.612613 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fca8195-a678-4212-83b7-1a230a4f2039 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:09:31.614086 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fca8195-a678-4212-83b7-1a230a4f2039 service neutron] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:09:31.625808 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1418/2810] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 16:09:31 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:09:36.265783 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb1afe1-8fd8-4514-85e5-bc85c7ed20ee None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:09:36.266250 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb1afe1-8fd8-4514-85e5-bc85c7ed20ee None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:09:36.266380 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb1afe1-8fd8-4514-85e5-bc85c7ed20ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:09:36.285447 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-8cb1afe1-8fd8-4514-85e5-bc85c7ed20ee None None] MFA Rules not processed for user `1cd3f1cbe0eb496f8e4c0692366fdefb`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:09:36.286642 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-8cb1afe1-8fd8-4514-85e5-bc85c7ed20ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:09:36.342770 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1393/2811] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 16:09:36 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:09:41.620600 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=6 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:09:46.334604 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=31 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:10:26.862829 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fca8195-a678-4212-83b7-1a230a4f2039 service neutron] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:10:26.872105 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fca8195-a678-4212-83b7-1a230a4f2039 service neutron] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:10:26.876299 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0062384c-62c5-4869-889f-c5b32449b488 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f4157105f3488f807b7475008ab722', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:10:26.876908 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0062384c-62c5-4869-889f-c5b32449b488 service neutron] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:10:26.877234 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0062384c-62c5-4869-889f-c5b32449b488 service neutron] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:10:26.877511 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0062384c-62c5-4869-889f-c5b32449b488 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:10:26.879773 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-0062384c-62c5-4869-889f-c5b32449b488 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:10:26.890650 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0062384c-62c5-4869-889f-c5b32449b488 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:10:26.892092 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0062384c-62c5-4869-889f-c5b32449b488 service neutron] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:10:26.901164 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1419/2812] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 16:10:26 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:10:36.897558 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=6 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:10:51.763271 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb1afe1-8fd8-4514-85e5-bc85c7ed20ee None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:10:51.764883 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-8cb1afe1-8fd8-4514-85e5-bc85c7ed20ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:10:51.798102 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb1afe1-8fd8-4514-85e5-bc85c7ed20ee None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:10:51.814283 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7749d7a7-4353-4d9a-9b24-873010c8b879 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a1e756ae9be4a7bac1e81acf07e8f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:10:51.815399 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7749d7a7-4353-4d9a-9b24-873010c8b879 service nova] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:10:51.815802 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7749d7a7-4353-4d9a-9b24-873010c8b879 service nova] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:10:51.816128 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7749d7a7-4353-4d9a-9b24-873010c8b879 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:10:51.819126 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-7749d7a7-4353-4d9a-9b24-873010c8b879 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:10:51.836266 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7749d7a7-4353-4d9a-9b24-873010c8b879 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:10:51.838851 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7749d7a7-4353-4d9a-9b24-873010c8b879 service nova] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:10:51.852950 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1394/2813] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 16:10:51 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:10:53.574341 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22875411-eb01-4698-a14e-dd89437f84b7 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:10:53.574893 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22875411-eb01-4698-a14e-dd89437f84b7 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:10:53.574893 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22875411-eb01-4698-a14e-dd89437f84b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:10:53.595357 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-22875411-eb01-4698-a14e-dd89437f84b7 None None] MFA Rules not processed for user `02287f232cdd4842a747bb749b483cc3`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:10:53.596765 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-22875411-eb01-4698-a14e-dd89437f84b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:10:53.609382 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1420/2814] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 16:10:53 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:10:54.391259 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7749d7a7-4353-4d9a-9b24-873010c8b879 service nova] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:10:54.392830 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-7749d7a7-4353-4d9a-9b24-873010c8b879 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:10:54.469017 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7749d7a7-4353-4d9a-9b24-873010c8b879 service nova] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:10:54.474426 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fdcae4-eb97-4b20-9ed4-ade2c4d9e3d7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc12c6d31eab4a17a160250adc2380c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:10:54.475028 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fdcae4-eb97-4b20-9ed4-ade2c4d9e3d7 service placement] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:10:54.475296 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fdcae4-eb97-4b20-9ed4-ade2c4d9e3d7 service placement] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:10:54.475560 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fdcae4-eb97-4b20-9ed4-ade2c4d9e3d7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:10:54.477334 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-05fdcae4-eb97-4b20-9ed4-ade2c4d9e3d7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:10:54.488934 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05fdcae4-eb97-4b20-9ed4-ade2c4d9e3d7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:10:54.490604 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05fdcae4-eb97-4b20-9ed4-ade2c4d9e3d7 service placement] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:10:54.500649 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1395/2815] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 16:10:54 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:11:01.139877 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22875411-eb01-4698-a14e-dd89437f84b7 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:11:01.141514 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-22875411-eb01-4698-a14e-dd89437f84b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:11:01.212868 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22875411-eb01-4698-a14e-dd89437f84b7 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:11:01.217711 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae785dc-3dc5-4fa9-923f-ec0fe112627c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:11:01.218336 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae785dc-3dc5-4fa9-923f-ec0fe112627c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:11:01.218637 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae785dc-3dc5-4fa9-923f-ec0fe112627c admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:11:01.218902 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae785dc-3dc5-4fa9-923f-ec0fe112627c admin admin] PATH_INFO: `/v3/users/1cd3f1cbe0eb496f8e4c0692366fdefb` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:11:01.220059 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae785dc-3dc5-4fa9-923f-ec0fe112627c admin admin] RBAC: Authorizing `identity:delete_user(user_id=1cd3f1cbe0eb496f8e4c0692366fdefb)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:11:01.222032 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae785dc-3dc5-4fa9-923f-ec0fe112627c admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:11:01.274752 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-3ae785dc-3dc5-4fa9-923f-ec0fe112627c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1cd3f1cbe0eb496f8e4c0692366fdefb'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:11:01.288639 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-3ae785dc-3dc5-4fa9-923f-ec0fe112627c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1cd3f1cbe0eb496f8e4c0692366fdefb'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:11:01.297906 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-3ae785dc-3dc5-4fa9-923f-ec0fe112627c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1cd3f1cbe0eb496f8e4c0692366fdefb'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:11:01.298616 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-3ae785dc-3dc5-4fa9-923f-ec0fe112627c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1cd3f1cbe0eb496f8e4c0692366fdefb'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:11:01.305746 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-3ae785dc-3dc5-4fa9-923f-ec0fe112627c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1cd3f1cbe0eb496f8e4c0692366fdefb'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:11:01.311640 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1421/2816] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Aug 30 16:11:01 2022] DELETE /identity/v3/users/1cd3f1cbe0eb496f8e4c0692366fdefb => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:11:01.500569 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fdcae4-eb97-4b20-9ed4-ade2c4d9e3d7 service placement] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:11:01.511672 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fdcae4-eb97-4b20-9ed4-ade2c4d9e3d7 service placement] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:11:01.574202 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f053e177-ace1-42e5-baf8-31e2c18a6aa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:11:01.574862 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f053e177-ace1-42e5-baf8-31e2c18a6aa0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:11:01.575139 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f053e177-ace1-42e5-baf8-31e2c18a6aa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:11:01.575468 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f053e177-ace1-42e5-baf8-31e2c18a6aa0 admin admin] PATH_INFO: `/v3/projects/12d8ded5bfda4c8ebb281a5504b56eb1` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:11:01.576623 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f053e177-ace1-42e5-baf8-31e2c18a6aa0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=12d8ded5bfda4c8ebb281a5504b56eb1)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:11:01.578579 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f053e177-ace1-42e5-baf8-31e2c18a6aa0 admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:11:01.628260 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-f053e177-ace1-42e5-baf8-31e2c18a6aa0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '12d8ded5bfda4c8ebb281a5504b56eb1', 'request_id': 'req-f053e177-ace1-42e5-baf8-31e2c18a6aa0'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:11:01.636417 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-f053e177-ace1-42e5-baf8-31e2c18a6aa0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '12d8ded5bfda4c8ebb281a5504b56eb1', 'request_id': 'req-f053e177-ace1-42e5-baf8-31e2c18a6aa0'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:11:01.643579 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-f053e177-ace1-42e5-baf8-31e2c18a6aa0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '12d8ded5bfda4c8ebb281a5504b56eb1', 'request_id': 'req-f053e177-ace1-42e5-baf8-31e2c18a6aa0'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:11:01.656534 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-f053e177-ace1-42e5-baf8-31e2c18a6aa0 admin admin] The token cache is being invalidate because project 12d8ded5bfda4c8ebb281a5504b56eb1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81512) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 16:11:01.656823 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-f053e177-ace1-42e5-baf8-31e2c18a6aa0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:11:01.657597 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-f053e177-ace1-42e5-baf8-31e2c18a6aa0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:11:01.659205 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1396/2817] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Aug 30 16:11:01 2022] DELETE /identity/v3/projects/12d8ded5bfda4c8ebb281a5504b56eb1 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:11:01.665450 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3fb945-1f96-4b25-8e17-9ff473085aa9 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:11:01.665721 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3fb945-1f96-4b25-8e17-9ff473085aa9 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:11:01.666002 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3fb945-1f96-4b25-8e17-9ff473085aa9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:11:01.689186 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-bd3fb945-1f96-4b25-8e17-9ff473085aa9 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:11:01.690438 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-bd3fb945-1f96-4b25-8e17-9ff473085aa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:11:01.753646 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1422/2818] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 16:11:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 16:11:01.763056 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d85781-e470-4cc2-81f4-ef2e4ed78050 None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:11:01.763337 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d85781-e470-4cc2-81f4-ef2e4ed78050 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:11:01.763610 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d85781-e470-4cc2-81f4-ef2e4ed78050 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:11:01.778522 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:11:01.784351 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-69d85781-e470-4cc2-81f4-ef2e4ed78050 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:11:01.786568 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-69d85781-e470-4cc2-81f4-ef2e4ed78050 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:11:01.795376 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1397/2819] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 16:11:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:11:01.799775 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3fb945-1f96-4b25-8e17-9ff473085aa9 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:11:01.810024 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3fb945-1f96-4b25-8e17-9ff473085aa9 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:11:01.815454 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffa610e-6ecf-4682-b57c-ac1bcb3c21a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:11:01.816051 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffa610e-6ecf-4682-b57c-ac1bcb3c21a8 admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:11:01.816324 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffa610e-6ecf-4682-b57c-ac1bcb3c21a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:11:01.816593 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffa610e-6ecf-4682-b57c-ac1bcb3c21a8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:11:01.817264 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ffa610e-6ecf-4682-b57c-ac1bcb3c21a8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:11:01.817583 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ffa610e-6ecf-4682-b57c-ac1bcb3c21a8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:11:01.819183 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ffa610e-6ecf-4682-b57c-ac1bcb3c21a8 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:11:01.828120 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:11:01.831878 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1423/2820] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Aug 30 16:11:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:11:01.837338 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d85781-e470-4cc2-81f4-ef2e4ed78050 None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:11:01.847038 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d85781-e470-4cc2-81f4-ef2e4ed78050 None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:11:01.853211 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3e5173-e5aa-4dca-aa74-df5d5b2220c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:11:01.853880 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3e5173-e5aa-4dca-aa74-df5d5b2220c6 admin admin] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:11:01.854156 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3e5173-e5aa-4dca-aa74-df5d5b2220c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:11:01.854420 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3e5173-e5aa-4dca-aa74-df5d5b2220c6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:11:01.855041 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3e5173-e5aa-4dca-aa74-df5d5b2220c6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:11:01.856942 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3e5173-e5aa-4dca-aa74-df5d5b2220c6 admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:11:01.896329 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1398/2821] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Aug 30 16:11:01 2022] POST /identity/v3/projects => generated 393 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:11:01.902974 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffa610e-6ecf-4682-b57c-ac1bcb3c21a8 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:11:01.917802 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffa610e-6ecf-4682-b57c-ac1bcb3c21a8 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:11:01.980312 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7873a40a-e4e7-4519-bc19-d03acb29d59a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:11:01.980914 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7873a40a-e4e7-4519-bc19-d03acb29d59a admin admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:11:01.981191 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7873a40a-e4e7-4519-bc19-d03acb29d59a admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:11:01.981519 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7873a40a-e4e7-4519-bc19-d03acb29d59a admin admin] PATH_INFO: `/v3/users` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:11:01.982115 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7873a40a-e4e7-4519-bc19-d03acb29d59a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:11:01.984030 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7873a40a-e4e7-4519-bc19-d03acb29d59a admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:11:02.004963 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1424/2822] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Aug 30 16:11:01 2022] POST /identity/v3/users => generated 297 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:11:02.009327 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3e5173-e5aa-4dca-aa74-df5d5b2220c6 admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:11:02.020550 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3e5173-e5aa-4dca-aa74-df5d5b2220c6 admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:11:02.027673 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b953847-f011-4a5e-805d-52c9478c1c9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:11:02.028788 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b953847-f011-4a5e-805d-52c9478c1c9e admin admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:11:02.029130 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b953847-f011-4a5e-805d-52c9478c1c9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:11:02.029589 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b953847-f011-4a5e-805d-52c9478c1c9e admin admin] PATH_INFO: `/v3/roles` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:11:02.030538 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b953847-f011-4a5e-805d-52c9478c1c9e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:11:02.032201 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b953847-f011-4a5e-805d-52c9478c1c9e admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:11:02.044199 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1399/2823] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 30 16:11:02 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:11:02.048042 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7873a40a-e4e7-4519-bc19-d03acb29d59a admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:11:02.058775 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7873a40a-e4e7-4519-bc19-d03acb29d59a admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:11:02.064413 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76cd65d-9fb4-4317-a744-ffea62f91f77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:11:02.065039 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76cd65d-9fb4-4317-a744-ffea62f91f77 admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:11:02.065353 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76cd65d-9fb4-4317-a744-ffea62f91f77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:11:02.065625 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76cd65d-9fb4-4317-a744-ffea62f91f77 admin admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:11:02.066267 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76cd65d-9fb4-4317-a744-ffea62f91f77 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:11:02.067852 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76cd65d-9fb4-4317-a744-ffea62f91f77 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:11:02.078814 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1425/2824] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 30 16:11:02 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:11:02.083058 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b953847-f011-4a5e-805d-52c9478c1c9e admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:11:02.093752 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b953847-f011-4a5e-805d-52c9478c1c9e admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:11:02.099140 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15d20de-e831-4a24-9521-32be85b36ecb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:11:02.099780 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15d20de-e831-4a24-9521-32be85b36ecb admin admin] REQUEST_METHOD: `PUT` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:11:02.100070 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15d20de-e831-4a24-9521-32be85b36ecb admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:11:02.100336 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15d20de-e831-4a24-9521-32be85b36ecb admin admin] PATH_INFO: `/v3/projects/d1a12fca19d6417890fd4fbfb1c9c31b/users/949cba01f8494832a0441ac1efbe6189/roles/df8f9f844b1b439181d75a07c1a93418` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:11:02.115042 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15d20de-e831-4a24-9521-32be85b36ecb admin admin] RBAC: Authorizing `identity:create_grant(project_id=d1a12fca19d6417890fd4fbfb1c9c31b, user_id=949cba01f8494832a0441ac1efbe6189, role_id=df8f9f844b1b439181d75a07c1a93418)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:11:02.117755 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15d20de-e831-4a24-9521-32be85b36ecb admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:11:02.132186 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1400/2825] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 30 16:11:02 2022] PUT /identity/v3/projects/d1a12fca19d6417890fd4fbfb1c9c31b/users/949cba01f8494832a0441ac1efbe6189/roles/df8f9f844b1b439181d75a07c1a93418 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:11:02.144898 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcfbddf-88e1-4956-8e8c-1f61b6d1ea10 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:11:02.144898 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcfbddf-88e1-4956-8e8c-1f61b6d1ea10 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:11:02.144898 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcfbddf-88e1-4956-8e8c-1f61b6d1ea10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:11:02.179748 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-bbcfbddf-88e1-4956-8e8c-1f61b6d1ea10 None None] MFA Rules not processed for user `949cba01f8494832a0441ac1efbe6189`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:11:02.181020 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-bbcfbddf-88e1-4956-8e8c-1f61b6d1ea10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:11:02.248299 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1426/2826] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 16:11:02 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:11:02.254629 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15d20de-e831-4a24-9521-32be85b36ecb admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:11:02.318657 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15d20de-e831-4a24-9521-32be85b36ecb admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:11:02.323971 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404caeb3-ac96-4fda-bff1-d6d6b0f5e189 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:11:02.324574 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404caeb3-ac96-4fda-bff1-d6d6b0f5e189 admin admin] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:11:02.324855 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404caeb3-ac96-4fda-bff1-d6d6b0f5e189 admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:11:02.325152 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404caeb3-ac96-4fda-bff1-d6d6b0f5e189 admin admin] PATH_INFO: `/v3/projects` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:11:02.325809 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404caeb3-ac96-4fda-bff1-d6d6b0f5e189 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:11:02.327780 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404caeb3-ac96-4fda-bff1-d6d6b0f5e189 admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:11:02.356693 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1401/2827] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Aug 30 16:11:02 2022] POST /identity/v3/projects => generated 393 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:11:02.361126 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcfbddf-88e1-4956-8e8c-1f61b6d1ea10 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:11:02.372186 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcfbddf-88e1-4956-8e8c-1f61b6d1ea10 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:11:02.429916 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81dd0f47-6bb7-4c9c-b7ad-b3173b5e0acc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:11:02.430516 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dd0f47-6bb7-4c9c-b7ad-b3173b5e0acc admin admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:11:02.430788 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dd0f47-6bb7-4c9c-b7ad-b3173b5e0acc admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:11:02.431181 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dd0f47-6bb7-4c9c-b7ad-b3173b5e0acc admin admin] PATH_INFO: `/v3/users` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:11:02.431849 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81dd0f47-6bb7-4c9c-b7ad-b3173b5e0acc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:11:02.433910 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81dd0f47-6bb7-4c9c-b7ad-b3173b5e0acc admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:11:02.452810 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:11:02.463810 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1427/2828] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Aug 30 16:11:02 2022] POST /identity/v3/users => generated 303 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:11:02.468861 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404caeb3-ac96-4fda-bff1-d6d6b0f5e189 admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:11:02.480246 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404caeb3-ac96-4fda-bff1-d6d6b0f5e189 admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:11:02.486561 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105ab2ee-8de6-4993-bb34-e06b0e0c6d40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:11:02.487154 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105ab2ee-8de6-4993-bb34-e06b0e0c6d40 admin admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:11:02.487432 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105ab2ee-8de6-4993-bb34-e06b0e0c6d40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:11:02.487701 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105ab2ee-8de6-4993-bb34-e06b0e0c6d40 admin admin] PATH_INFO: `/v3/roles` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:11:02.488339 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105ab2ee-8de6-4993-bb34-e06b0e0c6d40 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:11:02.490017 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105ab2ee-8de6-4993-bb34-e06b0e0c6d40 admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:11:02.501164 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1402/2829] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 30 16:11:02 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:11:02.505381 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81dd0f47-6bb7-4c9c-b7ad-b3173b5e0acc admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:11:02.516209 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81dd0f47-6bb7-4c9c-b7ad-b3173b5e0acc admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:11:02.521925 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43954469-3589-48d5-9494-ca256b9319a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:11:02.522578 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43954469-3589-48d5-9494-ca256b9319a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:11:02.522856 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43954469-3589-48d5-9494-ca256b9319a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:11:02.523126 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43954469-3589-48d5-9494-ca256b9319a4 admin admin] PATH_INFO: `/v3/projects/8daaf9406ffc45ee8dc7f4acd9fdf3ee/users/34745385fc2a4b54a461477affec37fa/roles/f265386950a641ac8413b53eda4ae34d` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:11:02.539984 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43954469-3589-48d5-9494-ca256b9319a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8daaf9406ffc45ee8dc7f4acd9fdf3ee, user_id=34745385fc2a4b54a461477affec37fa, role_id=f265386950a641ac8413b53eda4ae34d)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:11:02.542749 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43954469-3589-48d5-9494-ca256b9319a4 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:11:02.557862 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1428/2830] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 30 16:11:02 2022] PUT /identity/v3/projects/8daaf9406ffc45ee8dc7f4acd9fdf3ee/users/34745385fc2a4b54a461477affec37fa/roles/f265386950a641ac8413b53eda4ae34d => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 16:11:02.562172 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105ab2ee-8de6-4993-bb34-e06b0e0c6d40 admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:11:02.573637 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105ab2ee-8de6-4993-bb34-e06b0e0c6d40 admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:11:02.634269 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e482075-5df0-4b05-8f46-d3f42465e5ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:11:02.634951 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e482075-5df0-4b05-8f46-d3f42465e5ee admin admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:11:02.635230 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e482075-5df0-4b05-8f46-d3f42465e5ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:11:02.635504 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e482075-5df0-4b05-8f46-d3f42465e5ee admin admin] PATH_INFO: `/v3/roles` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:11:02.636153 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e482075-5df0-4b05-8f46-d3f42465e5ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:11:02.637913 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e482075-5df0-4b05-8f46-d3f42465e5ee admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:11:02.654692 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1403/2831] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 30 16:11:02 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:11:02.655358 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43954469-3589-48d5-9494-ca256b9319a4 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:11:02.666281 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43954469-3589-48d5-9494-ca256b9319a4 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:11:02.670922 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babb9e12-0e0e-42b2-93b2-6655fa3abcf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:11:02.671608 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babb9e12-0e0e-42b2-93b2-6655fa3abcf9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:11:02.671888 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babb9e12-0e0e-42b2-93b2-6655fa3abcf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:11:02.672156 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babb9e12-0e0e-42b2-93b2-6655fa3abcf9 admin admin] PATH_INFO: `/v3/projects/8daaf9406ffc45ee8dc7f4acd9fdf3ee/users/34745385fc2a4b54a461477affec37fa/roles/f265386950a641ac8413b53eda4ae34d` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:11:02.674117 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-babb9e12-0e0e-42b2-93b2-6655fa3abcf9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8daaf9406ffc45ee8dc7f4acd9fdf3ee, user_id=34745385fc2a4b54a461477affec37fa, role_id=f265386950a641ac8413b53eda4ae34d)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:11:02.676741 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-babb9e12-0e0e-42b2-93b2-6655fa3abcf9 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:11:02.687290 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1429/2832] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 30 16:11:02 2022] PUT /identity/v3/projects/8daaf9406ffc45ee8dc7f4acd9fdf3ee/users/34745385fc2a4b54a461477affec37fa/roles/f265386950a641ac8413b53eda4ae34d => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:11:02.698321 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88812783-7c4a-4f47-a035-5046bfa868b6 None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:11:02.698594 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88812783-7c4a-4f47-a035-5046bfa868b6 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:11:02.698883 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88812783-7c4a-4f47-a035-5046bfa868b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:11:02.707139 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:11:02.728887 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-88812783-7c4a-4f47-a035-5046bfa868b6 None None] MFA Rules not processed for user `34745385fc2a4b54a461477affec37fa`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:11:02.730282 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-88812783-7c4a-4f47-a035-5046bfa868b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:11:02.801851 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1404/2833] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 16:11:02 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:11:02.813021 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babb9e12-0e0e-42b2-93b2-6655fa3abcf9 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:11:02.815459 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-babb9e12-0e0e-42b2-93b2-6655fa3abcf9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:11:02.898748 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babb9e12-0e0e-42b2-93b2-6655fa3abcf9 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:11:02.902560 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9113d2a-e45b-47af-8f44-050978821ae5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0100a04821804993866ea20f159663f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:11:02.903163 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9113d2a-e45b-47af-8f44-050978821ae5 service cinder] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:11:02.903438 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9113d2a-e45b-47af-8f44-050978821ae5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:11:02.903700 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9113d2a-e45b-47af-8f44-050978821ae5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:11:02.912453 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9113d2a-e45b-47af-8f44-050978821ae5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:11:02.913909 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9113d2a-e45b-47af-8f44-050978821ae5 service cinder] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:11:02.923111 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1430/2834] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 16:11:02 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:11:02.969004 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88812783-7c4a-4f47-a035-5046bfa868b6 None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:11:02.970673 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-88812783-7c4a-4f47-a035-5046bfa868b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:11:02.985212 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88812783-7c4a-4f47-a035-5046bfa868b6 None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:11:02.989443 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7615606e-78d7-4d26-917f-b8e11c3f99d6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0100a04821804993866ea20f159663f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:11:02.990080 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7615606e-78d7-4d26-917f-b8e11c3f99d6 service cinder] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:11:02.990352 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7615606e-78d7-4d26-917f-b8e11c3f99d6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:11:02.990614 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7615606e-78d7-4d26-917f-b8e11c3f99d6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:11:03.066299 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7615606e-78d7-4d26-917f-b8e11c3f99d6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:11:03.067746 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7615606e-78d7-4d26-917f-b8e11c3f99d6 service cinder] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:11:03.076964 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1405/2835] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 16:11:02 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:11:06.468261 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9113d2a-e45b-47af-8f44-050978821ae5 service cinder] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:11:06.530373 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9113d2a-e45b-47af-8f44-050978821ae5 service cinder] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:11:06.536149 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477275b6-67f7-42b3-8a67-6d98bfe0924e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:11:06.536782 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477275b6-67f7-42b3-8a67-6d98bfe0924e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:11:06.537109 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477275b6-67f7-42b3-8a67-6d98bfe0924e admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:11:06.537423 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477275b6-67f7-42b3-8a67-6d98bfe0924e admin admin] PATH_INFO: `/v3/users/949cba01f8494832a0441ac1efbe6189` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:11:06.538683 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477275b6-67f7-42b3-8a67-6d98bfe0924e admin admin] RBAC: Authorizing `identity:delete_user(user_id=949cba01f8494832a0441ac1efbe6189)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:11:06.540671 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477275b6-67f7-42b3-8a67-6d98bfe0924e admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:11:06.588637 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:11:06.590279 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-477275b6-67f7-42b3-8a67-6d98bfe0924e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '949cba01f8494832a0441ac1efbe6189'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:11:06.604182 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-477275b6-67f7-42b3-8a67-6d98bfe0924e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '949cba01f8494832a0441ac1efbe6189'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:11:06.613172 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-477275b6-67f7-42b3-8a67-6d98bfe0924e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '949cba01f8494832a0441ac1efbe6189'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:11:06.614036 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-477275b6-67f7-42b3-8a67-6d98bfe0924e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '949cba01f8494832a0441ac1efbe6189'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:11:06.621084 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-477275b6-67f7-42b3-8a67-6d98bfe0924e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '949cba01f8494832a0441ac1efbe6189'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:11:06.627692 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1431/2836] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Aug 30 16:11:06 2022] DELETE /identity/v3/users/949cba01f8494832a0441ac1efbe6189 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:11:06.632379 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db0902a-15a3-4ac7-991f-b8b67b5ef56d None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:11:06.632971 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db0902a-15a3-4ac7-991f-b8b67b5ef56d None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:11:06.632971 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db0902a-15a3-4ac7-991f-b8b67b5ef56d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:11:06.653064 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-6db0902a-15a3-4ac7-991f-b8b67b5ef56d None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:11:06.654405 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-6db0902a-15a3-4ac7-991f-b8b67b5ef56d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:11:06.712533 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1406/2837] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 16:11:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:11:06.723625 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477275b6-67f7-42b3-8a67-6d98bfe0924e admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:11:06.725817 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-477275b6-67f7-42b3-8a67-6d98bfe0924e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:11:06.790498 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477275b6-67f7-42b3-8a67-6d98bfe0924e admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:11:06.794561 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4d5fc7-a7b6-45ce-8ed8-4f4cc779d8b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f4157105f3488f807b7475008ab722', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:11:06.795161 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4d5fc7-a7b6-45ce-8ed8-4f4cc779d8b0 service neutron] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:11:06.795435 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4d5fc7-a7b6-45ce-8ed8-4f4cc779d8b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:11:06.795704 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4d5fc7-a7b6-45ce-8ed8-4f4cc779d8b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:11:06.805887 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4d5fc7-a7b6-45ce-8ed8-4f4cc779d8b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:11:06.807330 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4d5fc7-a7b6-45ce-8ed8-4f4cc779d8b0 service neutron] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:11:06.817230 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1432/2838] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 16:11:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:11:07.090619 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db0902a-15a3-4ac7-991f-b8b67b5ef56d None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:11:07.092340 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-6db0902a-15a3-4ac7-991f-b8b67b5ef56d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:11:07.105854 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db0902a-15a3-4ac7-991f-b8b67b5ef56d None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:11:07.111065 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5930fb5-73a2-42e5-9244-23261c9666e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:11:07.111958 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5930fb5-73a2-42e5-9244-23261c9666e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:11:07.112318 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5930fb5-73a2-42e5-9244-23261c9666e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:11:07.112645 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5930fb5-73a2-42e5-9244-23261c9666e1 admin admin] PATH_INFO: `/v3/projects/d1a12fca19d6417890fd4fbfb1c9c31b` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:11:07.113891 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5930fb5-73a2-42e5-9244-23261c9666e1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d1a12fca19d6417890fd4fbfb1c9c31b)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:11:07.116849 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5930fb5-73a2-42e5-9244-23261c9666e1 admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:11:07.143091 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:11:07.169034 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-a5930fb5-73a2-42e5-9244-23261c9666e1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd1a12fca19d6417890fd4fbfb1c9c31b', 'request_id': 'req-a5930fb5-73a2-42e5-9244-23261c9666e1'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:11:07.177929 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-a5930fb5-73a2-42e5-9244-23261c9666e1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd1a12fca19d6417890fd4fbfb1c9c31b', 'request_id': 'req-a5930fb5-73a2-42e5-9244-23261c9666e1'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:11:07.184641 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-a5930fb5-73a2-42e5-9244-23261c9666e1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd1a12fca19d6417890fd4fbfb1c9c31b', 'request_id': 'req-a5930fb5-73a2-42e5-9244-23261c9666e1'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:11:07.195537 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-a5930fb5-73a2-42e5-9244-23261c9666e1 admin admin] The token cache is being invalidate because project d1a12fca19d6417890fd4fbfb1c9c31b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81512) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 16:11:07.195819 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-a5930fb5-73a2-42e5-9244-23261c9666e1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:11:07.196502 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-a5930fb5-73a2-42e5-9244-23261c9666e1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:11:07.198084 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1407/2839] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Aug 30 16:11:07 2022] DELETE /identity/v3/projects/d1a12fca19d6417890fd4fbfb1c9c31b => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:11:07.202445 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4d5fc7-a7b6-45ce-8ed8-4f4cc779d8b0 service neutron] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:11:07.213022 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4d5fc7-a7b6-45ce-8ed8-4f4cc779d8b0 service neutron] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:11:07.282890 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28b3992-3961-48c1-b791-ee9ed3892be5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:11:07.283520 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28b3992-3961-48c1-b791-ee9ed3892be5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:11:07.283792 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28b3992-3961-48c1-b791-ee9ed3892be5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:11:07.284057 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28b3992-3961-48c1-b791-ee9ed3892be5 admin admin] PATH_INFO: `/v3/users/34745385fc2a4b54a461477affec37fa` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:11:07.285200 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28b3992-3961-48c1-b791-ee9ed3892be5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=34745385fc2a4b54a461477affec37fa)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:11:07.287137 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28b3992-3961-48c1-b791-ee9ed3892be5 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:11:07.340727 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-a28b3992-3961-48c1-b791-ee9ed3892be5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '34745385fc2a4b54a461477affec37fa'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:11:07.353558 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-a28b3992-3961-48c1-b791-ee9ed3892be5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '34745385fc2a4b54a461477affec37fa'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:11:07.356162 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:11:07.363763 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-a28b3992-3961-48c1-b791-ee9ed3892be5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '34745385fc2a4b54a461477affec37fa'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:11:07.364659 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-a28b3992-3961-48c1-b791-ee9ed3892be5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '34745385fc2a4b54a461477affec37fa'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:11:07.375988 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-a28b3992-3961-48c1-b791-ee9ed3892be5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '34745385fc2a4b54a461477affec37fa'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:11:07.382397 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1433/2840] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Aug 30 16:11:07 2022] DELETE /identity/v3/users/34745385fc2a4b54a461477affec37fa => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:11:07.737930 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5930fb5-73a2-42e5-9244-23261c9666e1 admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:11:07.739518 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-a5930fb5-73a2-42e5-9244-23261c9666e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:11:07.758637 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5930fb5-73a2-42e5-9244-23261c9666e1 admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:11:07.816212 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4ee530-06c4-4710-b958-03fabfd0127e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:11:07.817001 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4ee530-06c4-4710-b958-03fabfd0127e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:11:07.817372 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4ee530-06c4-4710-b958-03fabfd0127e admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:11:07.817719 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4ee530-06c4-4710-b958-03fabfd0127e admin admin] PATH_INFO: `/v3/projects/8daaf9406ffc45ee8dc7f4acd9fdf3ee` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:11:07.819348 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4ee530-06c4-4710-b958-03fabfd0127e admin admin] RBAC: Authorizing `identity:delete_project(project_id=8daaf9406ffc45ee8dc7f4acd9fdf3ee)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:11:07.821244 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4ee530-06c4-4710-b958-03fabfd0127e admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:11:07.821647 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28b3992-3961-48c1-b791-ee9ed3892be5 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:11:07.823575 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-a28b3992-3961-48c1-b791-ee9ed3892be5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:11:07.879405 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-1f4ee530-06c4-4710-b958-03fabfd0127e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8daaf9406ffc45ee8dc7f4acd9fdf3ee', 'request_id': 'req-1f4ee530-06c4-4710-b958-03fabfd0127e'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:11:07.889940 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-1f4ee530-06c4-4710-b958-03fabfd0127e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8daaf9406ffc45ee8dc7f4acd9fdf3ee', 'request_id': 'req-1f4ee530-06c4-4710-b958-03fabfd0127e'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:11:07.897513 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-1f4ee530-06c4-4710-b958-03fabfd0127e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8daaf9406ffc45ee8dc7f4acd9fdf3ee', 'request_id': 'req-1f4ee530-06c4-4710-b958-03fabfd0127e'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:11:07.907425 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28b3992-3961-48c1-b791-ee9ed3892be5 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:11:07.910156 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-1f4ee530-06c4-4710-b958-03fabfd0127e admin admin] The token cache is being invalidate because project 8daaf9406ffc45ee8dc7f4acd9fdf3ee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81512) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 16:11:07.910396 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-1f4ee530-06c4-4710-b958-03fabfd0127e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:11:07.911179 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-1f4ee530-06c4-4710-b958-03fabfd0127e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:11:07.912653 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1408/2841] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Aug 30 16:11:07 2022] DELETE /identity/v3/projects/8daaf9406ffc45ee8dc7f4acd9fdf3ee => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:11:07.918023 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf76adf3-19ff-497d-b9bd-382a837a0d37 None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:11:07.918319 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf76adf3-19ff-497d-b9bd-382a837a0d37 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:11:07.918589 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf76adf3-19ff-497d-b9bd-382a837a0d37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:11:07.938940 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-cf76adf3-19ff-497d-b9bd-382a837a0d37 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:11:07.940189 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-cf76adf3-19ff-497d-b9bd-382a837a0d37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:11:07.973300 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71201c1b-056c-4acb-8033-994d675e1fe0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a1e756ae9be4a7bac1e81acf07e8f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:11:07.973914 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71201c1b-056c-4acb-8033-994d675e1fe0 service nova] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:11:07.974196 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71201c1b-056c-4acb-8033-994d675e1fe0 service nova] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:11:07.974465 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71201c1b-056c-4acb-8033-994d675e1fe0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:11:07.976323 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-71201c1b-056c-4acb-8033-994d675e1fe0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:11:07.979891 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:11:07.988504 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71201c1b-056c-4acb-8033-994d675e1fe0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:11:07.990087 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71201c1b-056c-4acb-8033-994d675e1fe0 service nova] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:11:08.000647 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1434/2842] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 16:11:07 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 16:11:08.002523 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1409/2843] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 16:11:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 16:11:08.011746 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc6f1dd-0127-4b17-b33e-73e6f099139b None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:11:08.012032 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc6f1dd-0127-4b17-b33e-73e6f099139b None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:11:08.012340 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc6f1dd-0127-4b17-b33e-73e6f099139b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:11:08.032919 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-edc6f1dd-0127-4b17-b33e-73e6f099139b None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:11:08.034421 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-edc6f1dd-0127-4b17-b33e-73e6f099139b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:11:08.042541 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1435/2844] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 16:11:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:11:08.047077 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf76adf3-19ff-497d-b9bd-382a837a0d37 None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:11:08.056979 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf76adf3-19ff-497d-b9bd-382a837a0d37 None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:11:08.061740 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34761066-66e9-400b-bc57-125da6d52f98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:11:08.062333 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34761066-66e9-400b-bc57-125da6d52f98 admin admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:11:08.062605 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34761066-66e9-400b-bc57-125da6d52f98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:11:08.062865 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34761066-66e9-400b-bc57-125da6d52f98 admin admin] PATH_INFO: `/v3/domains` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:11:08.063602 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34761066-66e9-400b-bc57-125da6d52f98 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81512) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:11:08.063905 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34761066-66e9-400b-bc57-125da6d52f98 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:11:08.065518 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34761066-66e9-400b-bc57-125da6d52f98 admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:11:08.078971 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1410/2845] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Aug 30 16:11:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:11:08.083628 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc6f1dd-0127-4b17-b33e-73e6f099139b None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:11:08.093262 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc6f1dd-0127-4b17-b33e-73e6f099139b None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:11:08.098370 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd506bb-6ebe-4c36-8d83-1bca5b3e2e92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:11:08.099055 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd506bb-6ebe-4c36-8d83-1bca5b3e2e92 admin admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:11:08.099332 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd506bb-6ebe-4c36-8d83-1bca5b3e2e92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:11:08.099629 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd506bb-6ebe-4c36-8d83-1bca5b3e2e92 admin admin] PATH_INFO: `/v3/projects` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:11:08.100274 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd506bb-6ebe-4c36-8d83-1bca5b3e2e92 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:11:08.102227 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd506bb-6ebe-4c36-8d83-1bca5b3e2e92 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:11:08.131764 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1436/2846] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Aug 30 16:11:08 2022] POST /identity/v3/projects => generated 385 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:11:08.136305 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34761066-66e9-400b-bc57-125da6d52f98 admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:11:08.147689 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34761066-66e9-400b-bc57-125da6d52f98 admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:11:08.217110 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3276749f-7ee2-4ef8-b1f4-c633f832755e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:11:08.217774 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3276749f-7ee2-4ef8-b1f4-c633f832755e admin admin] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:11:08.218055 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3276749f-7ee2-4ef8-b1f4-c633f832755e admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:11:08.218327 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3276749f-7ee2-4ef8-b1f4-c633f832755e admin admin] PATH_INFO: `/v3/users` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:11:08.218967 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3276749f-7ee2-4ef8-b1f4-c633f832755e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:11:08.220900 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3276749f-7ee2-4ef8-b1f4-c633f832755e admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:11:08.244453 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1411/2847] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Aug 30 16:11:08 2022] POST /identity/v3/users => generated 293 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:11:08.249863 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd506bb-6ebe-4c36-8d83-1bca5b3e2e92 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:11:08.260887 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd506bb-6ebe-4c36-8d83-1bca5b3e2e92 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:11:08.270103 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9692e67-e61c-4491-a192-e7d65911dda7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:11:08.270899 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9692e67-e61c-4491-a192-e7d65911dda7 admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:11:08.271180 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9692e67-e61c-4491-a192-e7d65911dda7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:11:08.271451 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9692e67-e61c-4491-a192-e7d65911dda7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:11:08.272114 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9692e67-e61c-4491-a192-e7d65911dda7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:11:08.273861 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9692e67-e61c-4491-a192-e7d65911dda7 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:11:08.286754 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1437/2848] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 30 16:11:08 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 16:11:08.292001 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3276749f-7ee2-4ef8-b1f4-c633f832755e admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:11:08.302946 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3276749f-7ee2-4ef8-b1f4-c633f832755e admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:11:08.308358 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fedd4e-23cc-4135-a829-de0f3ce1f5e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:11:08.309053 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fedd4e-23cc-4135-a829-de0f3ce1f5e0 admin admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:11:08.309361 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fedd4e-23cc-4135-a829-de0f3ce1f5e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:11:08.309631 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fedd4e-23cc-4135-a829-de0f3ce1f5e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:11:08.310325 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fedd4e-23cc-4135-a829-de0f3ce1f5e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:11:08.311930 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fedd4e-23cc-4135-a829-de0f3ce1f5e0 admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:11:08.322780 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1412/2849] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 30 16:11:08 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:11:08.327164 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9692e67-e61c-4491-a192-e7d65911dda7 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:11:08.338785 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9692e67-e61c-4491-a192-e7d65911dda7 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:11:08.344461 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e80c584-3bea-442b-b74a-121ded5f3e78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:11:08.345272 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e80c584-3bea-442b-b74a-121ded5f3e78 admin admin] REQUEST_METHOD: `PUT` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:11:08.345557 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e80c584-3bea-442b-b74a-121ded5f3e78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:11:08.345883 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e80c584-3bea-442b-b74a-121ded5f3e78 admin admin] PATH_INFO: `/v3/projects/108ba24a5e14423eb50ef11df8b65949/users/4f1cd3d9744d452ca66f534925831e8d/roles/df8f9f844b1b439181d75a07c1a93418` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:11:08.361083 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e80c584-3bea-442b-b74a-121ded5f3e78 admin admin] RBAC: Authorizing `identity:create_grant(project_id=108ba24a5e14423eb50ef11df8b65949, user_id=4f1cd3d9744d452ca66f534925831e8d, role_id=df8f9f844b1b439181d75a07c1a93418)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:11:08.363825 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e80c584-3bea-442b-b74a-121ded5f3e78 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:11:08.378361 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1438/2850] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 30 16:11:08 2022] PUT /identity/v3/projects/108ba24a5e14423eb50ef11df8b65949/users/4f1cd3d9744d452ca66f534925831e8d/roles/df8f9f844b1b439181d75a07c1a93418 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:11:08.388782 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3a2d01-cb1d-4843-9c91-bee1084cd6dc None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:11:08.389065 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3a2d01-cb1d-4843-9c91-bee1084cd6dc None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:11:08.389381 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3a2d01-cb1d-4843-9c91-bee1084cd6dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:11:08.417436 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-9c3a2d01-cb1d-4843-9c91-bee1084cd6dc None None] MFA Rules not processed for user `4f1cd3d9744d452ca66f534925831e8d`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:11:08.418693 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-9c3a2d01-cb1d-4843-9c91-bee1084cd6dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:11:08.484028 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1413/2851] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 16:11:08 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:11:08.488850 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e80c584-3bea-442b-b74a-121ded5f3e78 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:11:08.495469 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:DELETE=1,keystone:SELECT=97,keystone:INSERT=2 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:11:08.557496 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e80c584-3bea-442b-b74a-121ded5f3e78 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:11:08.563206 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3540958c-74b6-431d-b690-11e4918950fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:11:08.563819 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3540958c-74b6-431d-b690-11e4918950fe admin admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:11:08.564095 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3540958c-74b6-431d-b690-11e4918950fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:11:08.564390 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3540958c-74b6-431d-b690-11e4918950fe admin admin] PATH_INFO: `/v3/projects` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:11:08.565029 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3540958c-74b6-431d-b690-11e4918950fe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:11:08.567058 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3540958c-74b6-431d-b690-11e4918950fe admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:11:08.597442 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1439/2852] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Aug 30 16:11:08 2022] POST /identity/v3/projects => generated 385 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:11:08.602127 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3a2d01-cb1d-4843-9c91-bee1084cd6dc None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:11:08.613457 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3a2d01-cb1d-4843-9c91-bee1084cd6dc None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:11:08.648715 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:11:08.673093 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74db1c8-1e2d-4cdf-89ae-93333da9fb6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:11:08.673733 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74db1c8-1e2d-4cdf-89ae-93333da9fb6b admin admin] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:11:08.674005 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74db1c8-1e2d-4cdf-89ae-93333da9fb6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:11:08.674269 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74db1c8-1e2d-4cdf-89ae-93333da9fb6b admin admin] PATH_INFO: `/v3/users` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:11:08.674890 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74db1c8-1e2d-4cdf-89ae-93333da9fb6b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:11:08.676839 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74db1c8-1e2d-4cdf-89ae-93333da9fb6b admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:11:08.705605 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1414/2853] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Aug 30 16:11:08 2022] POST /identity/v3/users => generated 299 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:11:08.706523 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3540958c-74b6-431d-b690-11e4918950fe admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:11:08.716782 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3540958c-74b6-431d-b690-11e4918950fe admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:11:08.721625 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbd6634-a0e8-4b3f-92f0-2e737d3b3af1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:11:08.722231 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbd6634-a0e8-4b3f-92f0-2e737d3b3af1 admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:11:08.722500 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbd6634-a0e8-4b3f-92f0-2e737d3b3af1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:11:08.722773 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbd6634-a0e8-4b3f-92f0-2e737d3b3af1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:11:08.723761 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbd6634-a0e8-4b3f-92f0-2e737d3b3af1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:11:08.725367 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbd6634-a0e8-4b3f-92f0-2e737d3b3af1 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:11:08.741421 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1440/2854] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 30 16:11:08 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:11:08.746084 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74db1c8-1e2d-4cdf-89ae-93333da9fb6b admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:11:08.756532 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74db1c8-1e2d-4cdf-89ae-93333da9fb6b admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:11:08.761831 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83354ba9-0791-4b46-a737-d9fc80a7b685 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:11:08.762496 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83354ba9-0791-4b46-a737-d9fc80a7b685 admin admin] REQUEST_METHOD: `PUT` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:11:08.762782 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83354ba9-0791-4b46-a737-d9fc80a7b685 admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:11:08.763053 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83354ba9-0791-4b46-a737-d9fc80a7b685 admin admin] PATH_INFO: `/v3/projects/2b349dd623fa498b86462201a7b07a4a/users/de9519be73d2473bb5785989a32d6f55/roles/f265386950a641ac8413b53eda4ae34d` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:11:08.778543 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83354ba9-0791-4b46-a737-d9fc80a7b685 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2b349dd623fa498b86462201a7b07a4a, user_id=de9519be73d2473bb5785989a32d6f55, role_id=f265386950a641ac8413b53eda4ae34d)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:11:08.781245 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83354ba9-0791-4b46-a737-d9fc80a7b685 admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:11:08.796042 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1415/2855] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 30 16:11:08 2022] PUT /identity/v3/projects/2b349dd623fa498b86462201a7b07a4a/users/de9519be73d2473bb5785989a32d6f55/roles/f265386950a641ac8413b53eda4ae34d => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:11:08.800215 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbd6634-a0e8-4b3f-92f0-2e737d3b3af1 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:11:08.810485 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbd6634-a0e8-4b3f-92f0-2e737d3b3af1 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:11:08.868590 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2b98de-783d-43b8-953d-4c95740cf281 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:11:08.869201 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2b98de-783d-43b8-953d-4c95740cf281 admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:11:08.869572 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2b98de-783d-43b8-953d-4c95740cf281 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:11:08.869875 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2b98de-783d-43b8-953d-4c95740cf281 admin admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:11:08.870516 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2b98de-783d-43b8-953d-4c95740cf281 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:11:08.872096 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2b98de-783d-43b8-953d-4c95740cf281 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:11:08.881734 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1441/2856] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 30 16:11:08 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:11:08.886611 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83354ba9-0791-4b46-a737-d9fc80a7b685 admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:11:08.903403 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83354ba9-0791-4b46-a737-d9fc80a7b685 admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:11:08.909028 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dff0eec-e349-4715-9a0c-f39aeee4f214 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:11:08.909717 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dff0eec-e349-4715-9a0c-f39aeee4f214 admin admin] REQUEST_METHOD: `PUT` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:11:08.909997 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dff0eec-e349-4715-9a0c-f39aeee4f214 admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:11:08.910266 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dff0eec-e349-4715-9a0c-f39aeee4f214 admin admin] PATH_INFO: `/v3/projects/2b349dd623fa498b86462201a7b07a4a/users/de9519be73d2473bb5785989a32d6f55/roles/f265386950a641ac8413b53eda4ae34d` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:11:08.912365 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dff0eec-e349-4715-9a0c-f39aeee4f214 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2b349dd623fa498b86462201a7b07a4a, user_id=de9519be73d2473bb5785989a32d6f55, role_id=f265386950a641ac8413b53eda4ae34d)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:11:08.915105 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dff0eec-e349-4715-9a0c-f39aeee4f214 admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:11:08.926388 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1416/2857] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 30 16:11:08 2022] PUT /identity/v3/projects/2b349dd623fa498b86462201a7b07a4a/users/de9519be73d2473bb5785989a32d6f55/roles/f265386950a641ac8413b53eda4ae34d => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:11:08.937711 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50a02f0-6ab9-485a-9905-e6e00ea884d2 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:11:08.937996 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50a02f0-6ab9-485a-9905-e6e00ea884d2 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:11:08.938287 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50a02f0-6ab9-485a-9905-e6e00ea884d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:11:08.967452 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-f50a02f0-6ab9-485a-9905-e6e00ea884d2 None None] MFA Rules not processed for user `de9519be73d2473bb5785989a32d6f55`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:11:08.968654 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-f50a02f0-6ab9-485a-9905-e6e00ea884d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:11:09.036774 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:11:09.040642 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1442/2858] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 16:11:08 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:11:09.051691 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dff0eec-e349-4715-9a0c-f39aeee4f214 admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:11:09.053661 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-6dff0eec-e349-4715-9a0c-f39aeee4f214 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:11:09.116302 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dff0eec-e349-4715-9a0c-f39aeee4f214 admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:11:09.120557 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8a0a55-9f2a-4bfe-ba12-9f221e13c6e1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0100a04821804993866ea20f159663f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:11:09.121160 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8a0a55-9f2a-4bfe-ba12-9f221e13c6e1 service cinder] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:11:09.121471 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8a0a55-9f2a-4bfe-ba12-9f221e13c6e1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:11:09.121744 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8a0a55-9f2a-4bfe-ba12-9f221e13c6e1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:11:09.180043 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8a0a55-9f2a-4bfe-ba12-9f221e13c6e1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:11:09.181531 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8a0a55-9f2a-4bfe-ba12-9f221e13c6e1 service cinder] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:11:09.190372 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1417/2859] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 16:11:09 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:11:13.516418 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50a02f0-6ab9-485a-9905-e6e00ea884d2 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:11:13.517999 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-f50a02f0-6ab9-485a-9905-e6e00ea884d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:11:13.530971 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50a02f0-6ab9-485a-9905-e6e00ea884d2 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:11:13.534709 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479afe6a-07fc-47cd-8517-96dc823c020f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0100a04821804993866ea20f159663f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:11:13.535302 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479afe6a-07fc-47cd-8517-96dc823c020f service cinder] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:11:13.535572 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479afe6a-07fc-47cd-8517-96dc823c020f service cinder] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:11:13.535836 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479afe6a-07fc-47cd-8517-96dc823c020f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:11:13.545369 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479afe6a-07fc-47cd-8517-96dc823c020f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:11:13.546867 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479afe6a-07fc-47cd-8517-96dc823c020f service cinder] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:11:13.556328 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1443/2860] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 16:11:13 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:11:18.518592 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8a0a55-9f2a-4bfe-ba12-9f221e13c6e1 service cinder] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:11:18.565884 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:11:18.585810 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8a0a55-9f2a-4bfe-ba12-9f221e13c6e1 service cinder] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:11:18.592583 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42862ae-49cf-42ee-96ab-6d691aa8ce95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:11:18.593208 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42862ae-49cf-42ee-96ab-6d691aa8ce95 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:11:18.593522 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42862ae-49cf-42ee-96ab-6d691aa8ce95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:11:18.593798 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42862ae-49cf-42ee-96ab-6d691aa8ce95 admin admin] PATH_INFO: `/v3/users/4f1cd3d9744d452ca66f534925831e8d` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:11:18.595107 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42862ae-49cf-42ee-96ab-6d691aa8ce95 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4f1cd3d9744d452ca66f534925831e8d)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:11:18.597057 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42862ae-49cf-42ee-96ab-6d691aa8ce95 admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:11:18.657566 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-e42862ae-49cf-42ee-96ab-6d691aa8ce95 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4f1cd3d9744d452ca66f534925831e8d'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:11:18.666598 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-e42862ae-49cf-42ee-96ab-6d691aa8ce95 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4f1cd3d9744d452ca66f534925831e8d'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:11:18.680975 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-e42862ae-49cf-42ee-96ab-6d691aa8ce95 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4f1cd3d9744d452ca66f534925831e8d'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:11:18.689608 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-e42862ae-49cf-42ee-96ab-6d691aa8ce95 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4f1cd3d9744d452ca66f534925831e8d'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:11:18.690295 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-e42862ae-49cf-42ee-96ab-6d691aa8ce95 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4f1cd3d9744d452ca66f534925831e8d'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:11:18.696768 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1418/2861] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Aug 30 16:11:18 2022] DELETE /identity/v3/users/4f1cd3d9744d452ca66f534925831e8d => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:11:18.701595 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c29acf-97a3-401d-b6e1-33462d388c70 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:11:18.702090 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c29acf-97a3-401d-b6e1-33462d388c70 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:11:18.702212 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c29acf-97a3-401d-b6e1-33462d388c70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:11:18.722267 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-57c29acf-97a3-401d-b6e1-33462d388c70 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:11:18.723809 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-57c29acf-97a3-401d-b6e1-33462d388c70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:11:18.783331 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1444/2862] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 16:11:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:11:18.793262 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42862ae-49cf-42ee-96ab-6d691aa8ce95 admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:11:18.794774 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-e42862ae-49cf-42ee-96ab-6d691aa8ce95 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:11:18.857527 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42862ae-49cf-42ee-96ab-6d691aa8ce95 admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:11:18.861594 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8988184f-ff9c-44ac-95dc-aaa284456c56 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f4157105f3488f807b7475008ab722', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:11:18.862232 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8988184f-ff9c-44ac-95dc-aaa284456c56 service neutron] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:11:18.862510 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8988184f-ff9c-44ac-95dc-aaa284456c56 service neutron] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:11:18.862779 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8988184f-ff9c-44ac-95dc-aaa284456c56 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:11:18.871955 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8988184f-ff9c-44ac-95dc-aaa284456c56 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:11:18.873477 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8988184f-ff9c-44ac-95dc-aaa284456c56 service neutron] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:11:18.882833 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1419/2863] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 16:11:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:11:19.190278 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c29acf-97a3-401d-b6e1-33462d388c70 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:11:19.191813 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-57c29acf-97a3-401d-b6e1-33462d388c70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:11:19.204095 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c29acf-97a3-401d-b6e1-33462d388c70 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:11:19.209131 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8a2d6d-9ebd-44e9-b0a8-013aaa81297a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:11:19.209883 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8a2d6d-9ebd-44e9-b0a8-013aaa81297a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:11:19.210162 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8a2d6d-9ebd-44e9-b0a8-013aaa81297a admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:11:19.210428 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8a2d6d-9ebd-44e9-b0a8-013aaa81297a admin admin] PATH_INFO: `/v3/projects/108ba24a5e14423eb50ef11df8b65949` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:11:19.212559 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8a2d6d-9ebd-44e9-b0a8-013aaa81297a admin admin] RBAC: Authorizing `identity:delete_project(project_id=108ba24a5e14423eb50ef11df8b65949)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:11:19.214607 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8a2d6d-9ebd-44e9-b0a8-013aaa81297a admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:11:19.267718 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-7f8a2d6d-9ebd-44e9-b0a8-013aaa81297a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '108ba24a5e14423eb50ef11df8b65949', 'request_id': 'req-7f8a2d6d-9ebd-44e9-b0a8-013aaa81297a'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:11:19.277020 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-7f8a2d6d-9ebd-44e9-b0a8-013aaa81297a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '108ba24a5e14423eb50ef11df8b65949', 'request_id': 'req-7f8a2d6d-9ebd-44e9-b0a8-013aaa81297a'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:11:19.283372 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-7f8a2d6d-9ebd-44e9-b0a8-013aaa81297a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '108ba24a5e14423eb50ef11df8b65949', 'request_id': 'req-7f8a2d6d-9ebd-44e9-b0a8-013aaa81297a'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:11:19.294288 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-7f8a2d6d-9ebd-44e9-b0a8-013aaa81297a admin admin] The token cache is being invalidate because project 108ba24a5e14423eb50ef11df8b65949 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81511) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 16:11:19.294535 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-7f8a2d6d-9ebd-44e9-b0a8-013aaa81297a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:11:19.295202 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-7f8a2d6d-9ebd-44e9-b0a8-013aaa81297a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:11:19.296732 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1445/2864] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Aug 30 16:11:19 2022] DELETE /identity/v3/projects/108ba24a5e14423eb50ef11df8b65949 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:11:19.301077 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8988184f-ff9c-44ac-95dc-aaa284456c56 service neutron] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:11:19.311198 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8988184f-ff9c-44ac-95dc-aaa284456c56 service neutron] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:11:19.378656 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d07e29f-37b4-4e5e-9e47-8e615add639f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:11:19.379281 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d07e29f-37b4-4e5e-9e47-8e615add639f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:11:19.379566 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d07e29f-37b4-4e5e-9e47-8e615add639f admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:11:19.379834 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d07e29f-37b4-4e5e-9e47-8e615add639f admin admin] PATH_INFO: `/v3/users/de9519be73d2473bb5785989a32d6f55` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:11:19.380993 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d07e29f-37b4-4e5e-9e47-8e615add639f admin admin] RBAC: Authorizing `identity:delete_user(user_id=de9519be73d2473bb5785989a32d6f55)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:11:19.382973 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d07e29f-37b4-4e5e-9e47-8e615add639f admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:11:19.388863 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:11:19.431743 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-6d07e29f-37b4-4e5e-9e47-8e615add639f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'de9519be73d2473bb5785989a32d6f55'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:11:19.441437 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-6d07e29f-37b4-4e5e-9e47-8e615add639f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'de9519be73d2473bb5785989a32d6f55'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:11:19.454319 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-6d07e29f-37b4-4e5e-9e47-8e615add639f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'de9519be73d2473bb5785989a32d6f55'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:11:19.462027 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-6d07e29f-37b4-4e5e-9e47-8e615add639f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'de9519be73d2473bb5785989a32d6f55'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:11:19.462772 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-6d07e29f-37b4-4e5e-9e47-8e615add639f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'de9519be73d2473bb5785989a32d6f55'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:11:19.469411 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1420/2865] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Aug 30 16:11:19 2022] DELETE /identity/v3/users/de9519be73d2473bb5785989a32d6f55 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:11:19.777984 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8a2d6d-9ebd-44e9-b0a8-013aaa81297a admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:11:19.779631 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-7f8a2d6d-9ebd-44e9-b0a8-013aaa81297a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:11:19.794535 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8a2d6d-9ebd-44e9-b0a8-013aaa81297a admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:11:19.854631 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893f88f1-4e73-4d6f-986d-e5e9097507f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:11:19.855191 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893f88f1-4e73-4d6f-986d-e5e9097507f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:11:19.855465 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893f88f1-4e73-4d6f-986d-e5e9097507f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:11:19.855756 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893f88f1-4e73-4d6f-986d-e5e9097507f0 admin admin] PATH_INFO: `/v3/projects/2b349dd623fa498b86462201a7b07a4a` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:11:19.857030 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893f88f1-4e73-4d6f-986d-e5e9097507f0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2b349dd623fa498b86462201a7b07a4a)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:11:19.859155 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893f88f1-4e73-4d6f-986d-e5e9097507f0 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:11:19.870991 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:11:19.909402 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-893f88f1-4e73-4d6f-986d-e5e9097507f0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2b349dd623fa498b86462201a7b07a4a', 'request_id': 'req-893f88f1-4e73-4d6f-986d-e5e9097507f0'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:11:19.918838 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-893f88f1-4e73-4d6f-986d-e5e9097507f0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2b349dd623fa498b86462201a7b07a4a', 'request_id': 'req-893f88f1-4e73-4d6f-986d-e5e9097507f0'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:11:19.925772 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-893f88f1-4e73-4d6f-986d-e5e9097507f0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2b349dd623fa498b86462201a7b07a4a', 'request_id': 'req-893f88f1-4e73-4d6f-986d-e5e9097507f0'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:11:19.936662 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-893f88f1-4e73-4d6f-986d-e5e9097507f0 admin admin] The token cache is being invalidate because project 2b349dd623fa498b86462201a7b07a4a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81511) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 16:11:19.936980 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-893f88f1-4e73-4d6f-986d-e5e9097507f0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:11:19.937724 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-893f88f1-4e73-4d6f-986d-e5e9097507f0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:11:19.939308 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1446/2866] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Aug 30 16:11:19 2022] DELETE /identity/v3/projects/2b349dd623fa498b86462201a7b07a4a => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:11:19.945643 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886c9146-6107-425b-ba31-73f265c62951 None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:11:19.946136 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886c9146-6107-425b-ba31-73f265c62951 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:11:19.946136 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886c9146-6107-425b-ba31-73f265c62951 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:11:19.967804 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-886c9146-6107-425b-ba31-73f265c62951 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:11:19.968995 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-886c9146-6107-425b-ba31-73f265c62951 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:11:20.039282 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1421/2867] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 16:11:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:11:20.048067 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb93d16-d602-4798-8309-cfb4361c3181 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:11:20.048361 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb93d16-d602-4798-8309-cfb4361c3181 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:11:20.048631 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb93d16-d602-4798-8309-cfb4361c3181 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:11:20.069567 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-dcb93d16-d602-4798-8309-cfb4361c3181 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:11:20.070787 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-dcb93d16-d602-4798-8309-cfb4361c3181 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:11:20.079671 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1447/2868] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 16:11:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:11:20.084243 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886c9146-6107-425b-ba31-73f265c62951 None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:11:20.095186 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886c9146-6107-425b-ba31-73f265c62951 None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:11:20.101254 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1f9bb4-c1de-4c4a-9156-85c8ef5b27cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:11:20.101858 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1f9bb4-c1de-4c4a-9156-85c8ef5b27cd admin admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:11:20.102129 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1f9bb4-c1de-4c4a-9156-85c8ef5b27cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:11:20.102397 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1f9bb4-c1de-4c4a-9156-85c8ef5b27cd admin admin] PATH_INFO: `/v3/domains` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:11:20.103040 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1f9bb4-c1de-4c4a-9156-85c8ef5b27cd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81512) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:11:20.103341 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1f9bb4-c1de-4c4a-9156-85c8ef5b27cd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:11:20.105442 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1f9bb4-c1de-4c4a-9156-85c8ef5b27cd admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:11:20.119757 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1422/2869] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Aug 30 16:11:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:11:20.124568 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb93d16-d602-4798-8309-cfb4361c3181 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:11:20.135308 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb93d16-d602-4798-8309-cfb4361c3181 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:11:20.141729 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2afb07e-a0c9-4768-97ca-84665a2214e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:11:20.142345 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2afb07e-a0c9-4768-97ca-84665a2214e0 admin admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:11:20.142618 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2afb07e-a0c9-4768-97ca-84665a2214e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:11:20.142884 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2afb07e-a0c9-4768-97ca-84665a2214e0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:11:20.143522 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2afb07e-a0c9-4768-97ca-84665a2214e0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:11:20.145458 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2afb07e-a0c9-4768-97ca-84665a2214e0 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:11:20.175385 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1448/2870] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Aug 30 16:11:20 2022] POST /identity/v3/projects => generated 375 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:11:20.182086 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1f9bb4-c1de-4c4a-9156-85c8ef5b27cd admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:11:20.194801 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1f9bb4-c1de-4c4a-9156-85c8ef5b27cd admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:11:20.259245 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e2d455-0a48-40f3-80cb-dd19e71ee09e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:11:20.259883 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e2d455-0a48-40f3-80cb-dd19e71ee09e admin admin] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:11:20.260225 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e2d455-0a48-40f3-80cb-dd19e71ee09e admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:11:20.260500 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e2d455-0a48-40f3-80cb-dd19e71ee09e admin admin] PATH_INFO: `/v3/users` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:11:20.261121 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e2d455-0a48-40f3-80cb-dd19e71ee09e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:11:20.263468 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e2d455-0a48-40f3-80cb-dd19e71ee09e admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:11:20.287385 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1423/2871] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Aug 30 16:11:20 2022] POST /identity/v3/users => generated 288 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 16:11:20.291830 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2afb07e-a0c9-4768-97ca-84665a2214e0 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:11:20.306716 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2afb07e-a0c9-4768-97ca-84665a2214e0 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:11:20.315153 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d165999-4e34-40ee-9cf0-4e72321f677c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:11:20.316192 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d165999-4e34-40ee-9cf0-4e72321f677c admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:11:20.316695 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d165999-4e34-40ee-9cf0-4e72321f677c admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:11:20.317181 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d165999-4e34-40ee-9cf0-4e72321f677c admin admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:11:20.318251 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d165999-4e34-40ee-9cf0-4e72321f677c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:11:20.320306 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d165999-4e34-40ee-9cf0-4e72321f677c admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:11:20.336882 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1449/2872] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 30 16:11:20 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:11:20.343139 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e2d455-0a48-40f3-80cb-dd19e71ee09e admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:11:20.349166 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:11:20.354674 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e2d455-0a48-40f3-80cb-dd19e71ee09e admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:11:20.360269 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d94e4aa-f064-4c1e-b830-ef211d38b561 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:11:20.360937 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d94e4aa-f064-4c1e-b830-ef211d38b561 admin admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:11:20.361584 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d94e4aa-f064-4c1e-b830-ef211d38b561 admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:11:20.361896 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d94e4aa-f064-4c1e-b830-ef211d38b561 admin admin] PATH_INFO: `/v3/roles` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:11:20.362569 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d94e4aa-f064-4c1e-b830-ef211d38b561 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:11:20.364334 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d94e4aa-f064-4c1e-b830-ef211d38b561 admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:11:20.375377 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1424/2873] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 30 16:11:20 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:11:20.379558 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d165999-4e34-40ee-9cf0-4e72321f677c admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:11:20.389978 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d165999-4e34-40ee-9cf0-4e72321f677c admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:11:20.395193 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6204d5-cd2d-40f8-86ed-76005a682784 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:11:20.395839 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6204d5-cd2d-40f8-86ed-76005a682784 admin admin] REQUEST_METHOD: `PUT` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:11:20.396112 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6204d5-cd2d-40f8-86ed-76005a682784 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:11:20.396388 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6204d5-cd2d-40f8-86ed-76005a682784 admin admin] PATH_INFO: `/v3/projects/53b7d5d205c5467686eca774b393763e/users/add37998bcb343cc99166a5d9a3b9940/roles/df8f9f844b1b439181d75a07c1a93418` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:11:20.411314 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6204d5-cd2d-40f8-86ed-76005a682784 admin admin] RBAC: Authorizing `identity:create_grant(project_id=53b7d5d205c5467686eca774b393763e, user_id=add37998bcb343cc99166a5d9a3b9940, role_id=df8f9f844b1b439181d75a07c1a93418)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:11:20.414015 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6204d5-cd2d-40f8-86ed-76005a682784 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:11:20.431369 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1450/2874] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 30 16:11:20 2022] PUT /identity/v3/projects/53b7d5d205c5467686eca774b393763e/users/add37998bcb343cc99166a5d9a3b9940/roles/df8f9f844b1b439181d75a07c1a93418 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:11:20.442784 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6d0272-8d77-401e-a587-bb658577db3b None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:11:20.443072 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6d0272-8d77-401e-a587-bb658577db3b None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:11:20.443363 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6d0272-8d77-401e-a587-bb658577db3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:11:20.472080 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-3a6d0272-8d77-401e-a587-bb658577db3b None None] MFA Rules not processed for user `add37998bcb343cc99166a5d9a3b9940`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:11:20.473695 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-3a6d0272-8d77-401e-a587-bb658577db3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:11:20.544418 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1425/2875] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 16:11:20 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:11:20.560797 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6204d5-cd2d-40f8-86ed-76005a682784 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:11:20.563822 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-5b6204d5-cd2d-40f8-86ed-76005a682784 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:11:20.628842 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6204d5-cd2d-40f8-86ed-76005a682784 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:11:20.632705 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e1ed6a-e8eb-43b7-bdff-e44702a76c46 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0100a04821804993866ea20f159663f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:11:20.633352 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e1ed6a-e8eb-43b7-bdff-e44702a76c46 service cinder] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:11:20.633631 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e1ed6a-e8eb-43b7-bdff-e44702a76c46 service cinder] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:11:20.633897 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e1ed6a-e8eb-43b7-bdff-e44702a76c46 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:11:20.643092 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e1ed6a-e8eb-43b7-bdff-e44702a76c46 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:11:20.644556 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e1ed6a-e8eb-43b7-bdff-e44702a76c46 service cinder] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:11:20.653604 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1451/2876] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 16:11:20 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:11:23.745592 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6d0272-8d77-401e-a587-bb658577db3b None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:11:23.814074 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6d0272-8d77-401e-a587-bb658577db3b None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:11:23.819530 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac511d2-16c7-4231-9e42-d7835ad18f12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:11:23.820152 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac511d2-16c7-4231-9e42-d7835ad18f12 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:11:23.820433 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac511d2-16c7-4231-9e42-d7835ad18f12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:11:23.820710 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac511d2-16c7-4231-9e42-d7835ad18f12 admin admin] PATH_INFO: `/v3/users/add37998bcb343cc99166a5d9a3b9940` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:11:23.821939 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac511d2-16c7-4231-9e42-d7835ad18f12 admin admin] RBAC: Authorizing `identity:delete_user(user_id=add37998bcb343cc99166a5d9a3b9940)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:11:23.823877 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac511d2-16c7-4231-9e42-d7835ad18f12 admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:11:23.885456 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-2ac511d2-16c7-4231-9e42-d7835ad18f12 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'add37998bcb343cc99166a5d9a3b9940'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:11:23.894598 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-2ac511d2-16c7-4231-9e42-d7835ad18f12 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'add37998bcb343cc99166a5d9a3b9940'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:11:23.906586 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-2ac511d2-16c7-4231-9e42-d7835ad18f12 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'add37998bcb343cc99166a5d9a3b9940'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:11:23.913143 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-2ac511d2-16c7-4231-9e42-d7835ad18f12 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'add37998bcb343cc99166a5d9a3b9940'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:11:23.913849 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-2ac511d2-16c7-4231-9e42-d7835ad18f12 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'add37998bcb343cc99166a5d9a3b9940'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:11:23.916077 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:11:23.920561 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1426/2877] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Aug 30 16:11:23 2022] DELETE /identity/v3/users/add37998bcb343cc99166a5d9a3b9940 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:11:23.925245 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c347ba86-52f4-41de-8c7c-7edb5fe28797 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:11:23.925830 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c347ba86-52f4-41de-8c7c-7edb5fe28797 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:11:23.925830 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c347ba86-52f4-41de-8c7c-7edb5fe28797 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:11:23.945929 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-c347ba86-52f4-41de-8c7c-7edb5fe28797 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:11:23.947145 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-c347ba86-52f4-41de-8c7c-7edb5fe28797 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:11:23.982894 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:11:24.009837 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1452/2878] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 16:11:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:11:24.020022 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac511d2-16c7-4231-9e42-d7835ad18f12 admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:11:24.021566 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-2ac511d2-16c7-4231-9e42-d7835ad18f12 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:11:24.091312 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac511d2-16c7-4231-9e42-d7835ad18f12 admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:11:24.095369 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23be65d-98cd-45fe-b484-41d0384b96a9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f4157105f3488f807b7475008ab722', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:11:24.096004 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23be65d-98cd-45fe-b484-41d0384b96a9 service neutron] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:11:24.096281 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23be65d-98cd-45fe-b484-41d0384b96a9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:11:24.096524 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23be65d-98cd-45fe-b484-41d0384b96a9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:11:24.106442 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23be65d-98cd-45fe-b484-41d0384b96a9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:11:24.107900 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23be65d-98cd-45fe-b484-41d0384b96a9 service neutron] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:11:24.117570 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1427/2879] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 16:11:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:11:24.395908 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c347ba86-52f4-41de-8c7c-7edb5fe28797 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:11:24.397614 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-c347ba86-52f4-41de-8c7c-7edb5fe28797 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:11:24.410119 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c347ba86-52f4-41de-8c7c-7edb5fe28797 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:11:24.415117 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3a82b9-8e26-4f75-9ec4-c6ee74adbe10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:11:24.415827 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3a82b9-8e26-4f75-9ec4-c6ee74adbe10 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:11:24.416209 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3a82b9-8e26-4f75-9ec4-c6ee74adbe10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:11:24.416474 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3a82b9-8e26-4f75-9ec4-c6ee74adbe10 admin admin] PATH_INFO: `/v3/projects/53b7d5d205c5467686eca774b393763e` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:11:24.418524 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3a82b9-8e26-4f75-9ec4-c6ee74adbe10 admin admin] RBAC: Authorizing `identity:delete_project(project_id=53b7d5d205c5467686eca774b393763e)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:11:24.421128 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3a82b9-8e26-4f75-9ec4-c6ee74adbe10 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:11:24.476928 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-fa3a82b9-8e26-4f75-9ec4-c6ee74adbe10 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '53b7d5d205c5467686eca774b393763e', 'request_id': 'req-fa3a82b9-8e26-4f75-9ec4-c6ee74adbe10'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:11:24.486636 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-fa3a82b9-8e26-4f75-9ec4-c6ee74adbe10 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '53b7d5d205c5467686eca774b393763e', 'request_id': 'req-fa3a82b9-8e26-4f75-9ec4-c6ee74adbe10'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:11:24.493134 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-fa3a82b9-8e26-4f75-9ec4-c6ee74adbe10 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '53b7d5d205c5467686eca774b393763e', 'request_id': 'req-fa3a82b9-8e26-4f75-9ec4-c6ee74adbe10'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:11:24.503812 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-fa3a82b9-8e26-4f75-9ec4-c6ee74adbe10 admin admin] The token cache is being invalidate because project 53b7d5d205c5467686eca774b393763e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81511) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 16:11:24.504095 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-fa3a82b9-8e26-4f75-9ec4-c6ee74adbe10 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:11:24.504728 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-fa3a82b9-8e26-4f75-9ec4-c6ee74adbe10 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:11:24.506268 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1453/2880] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Aug 30 16:11:24 2022] DELETE /identity/v3/projects/53b7d5d205c5467686eca774b393763e => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:11:24.511215 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197733e4-fb5e-4049-ba4b-37fbd7d364bd None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:11:24.511498 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197733e4-fb5e-4049-ba4b-37fbd7d364bd None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:11:24.511768 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197733e4-fb5e-4049-ba4b-37fbd7d364bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:11:24.531183 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-197733e4-fb5e-4049-ba4b-37fbd7d364bd None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:11:24.532393 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-197733e4-fb5e-4049-ba4b-37fbd7d364bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:11:24.596005 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1428/2881] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 16:11:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:11:24.662586 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5554449-5e62-4046-b1b3-0ad21a5e174b None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:11:24.662825 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5554449-5e62-4046-b1b3-0ad21a5e174b None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:11:24.663126 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5554449-5e62-4046-b1b3-0ad21a5e174b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:11:24.684196 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-d5554449-5e62-4046-b1b3-0ad21a5e174b None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:11:24.685455 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-d5554449-5e62-4046-b1b3-0ad21a5e174b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:11:24.694276 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1454/2882] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 16:11:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:11:24.698758 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197733e4-fb5e-4049-ba4b-37fbd7d364bd None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:11:24.708749 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197733e4-fb5e-4049-ba4b-37fbd7d364bd None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:11:24.714316 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb67990-f8b3-4b1a-bf1d-6429fb44bd09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:11:24.714921 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb67990-f8b3-4b1a-bf1d-6429fb44bd09 admin admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:11:24.715192 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb67990-f8b3-4b1a-bf1d-6429fb44bd09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:11:24.715574 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb67990-f8b3-4b1a-bf1d-6429fb44bd09 admin admin] PATH_INFO: `/v3/domains` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:11:24.716226 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb67990-f8b3-4b1a-bf1d-6429fb44bd09 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81512) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:11:24.716526 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb67990-f8b3-4b1a-bf1d-6429fb44bd09 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:11:24.718174 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb67990-f8b3-4b1a-bf1d-6429fb44bd09 admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:11:24.730749 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1429/2883] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Aug 30 16:11:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 16:11:24.735163 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5554449-5e62-4046-b1b3-0ad21a5e174b None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:11:24.745068 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5554449-5e62-4046-b1b3-0ad21a5e174b None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:11:24.750883 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0f3312-e92a-4c6b-a8cb-291ce12795e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:11:24.751459 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0f3312-e92a-4c6b-a8cb-291ce12795e6 admin admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:11:24.751772 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0f3312-e92a-4c6b-a8cb-291ce12795e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:11:24.752029 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0f3312-e92a-4c6b-a8cb-291ce12795e6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:11:24.752645 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0f3312-e92a-4c6b-a8cb-291ce12795e6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:11:24.754648 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0f3312-e92a-4c6b-a8cb-291ce12795e6 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:11:24.784312 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1455/2884] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Aug 30 16:11:24 2022] POST /identity/v3/projects => generated 373 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:11:24.790311 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb67990-f8b3-4b1a-bf1d-6429fb44bd09 admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:11:24.804175 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb67990-f8b3-4b1a-bf1d-6429fb44bd09 admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:11:24.864822 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7104790-9754-423f-89e4-703a1194b202 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:11:24.865452 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7104790-9754-423f-89e4-703a1194b202 admin admin] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:11:24.865730 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7104790-9754-423f-89e4-703a1194b202 admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:11:24.865999 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7104790-9754-423f-89e4-703a1194b202 admin admin] PATH_INFO: `/v3/users` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:11:24.866614 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7104790-9754-423f-89e4-703a1194b202 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:11:24.868573 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7104790-9754-423f-89e4-703a1194b202 admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:11:24.894865 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1430/2885] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Aug 30 16:11:24 2022] POST /identity/v3/users => generated 287 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 16:11:24.899310 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0f3312-e92a-4c6b-a8cb-291ce12795e6 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:11:24.910291 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0f3312-e92a-4c6b-a8cb-291ce12795e6 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:11:24.915849 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0273f983-dc60-4131-840e-36dcc20441ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:11:24.916458 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0273f983-dc60-4131-840e-36dcc20441ab admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:11:24.916732 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0273f983-dc60-4131-840e-36dcc20441ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:11:24.917002 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0273f983-dc60-4131-840e-36dcc20441ab admin admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:11:24.917696 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0273f983-dc60-4131-840e-36dcc20441ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:11:24.919320 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0273f983-dc60-4131-840e-36dcc20441ab admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:11:24.929737 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1456/2886] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 30 16:11:24 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:11:24.934126 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7104790-9754-423f-89e4-703a1194b202 admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:11:24.944842 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7104790-9754-423f-89e4-703a1194b202 admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:11:24.950129 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fddd54-dbf5-41b6-8f89-0f48b6f6a6e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:11:24.950721 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fddd54-dbf5-41b6-8f89-0f48b6f6a6e0 admin admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:11:24.951180 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fddd54-dbf5-41b6-8f89-0f48b6f6a6e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:11:24.951391 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fddd54-dbf5-41b6-8f89-0f48b6f6a6e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:11:24.952039 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7fddd54-dbf5-41b6-8f89-0f48b6f6a6e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:11:24.953689 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7fddd54-dbf5-41b6-8f89-0f48b6f6a6e0 admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:11:24.964371 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1431/2887] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 30 16:11:24 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:11:24.968890 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0273f983-dc60-4131-840e-36dcc20441ab admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:11:24.979294 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0273f983-dc60-4131-840e-36dcc20441ab admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:11:24.984159 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77111c25-53a8-4456-90ad-926676bc5596 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:11:24.984748 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77111c25-53a8-4456-90ad-926676bc5596 admin admin] REQUEST_METHOD: `PUT` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:11:24.985027 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77111c25-53a8-4456-90ad-926676bc5596 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:11:24.985336 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77111c25-53a8-4456-90ad-926676bc5596 admin admin] PATH_INFO: `/v3/projects/1adbee870d654476a06ae02744a7ed95/users/ba852aa293f74f639369b6d224903aa1/roles/df8f9f844b1b439181d75a07c1a93418` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:11:24.999819 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77111c25-53a8-4456-90ad-926676bc5596 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1adbee870d654476a06ae02744a7ed95, user_id=ba852aa293f74f639369b6d224903aa1, role_id=df8f9f844b1b439181d75a07c1a93418)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:11:25.002544 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77111c25-53a8-4456-90ad-926676bc5596 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:11:25.017655 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1457/2888] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 30 16:11:24 2022] PUT /identity/v3/projects/1adbee870d654476a06ae02744a7ed95/users/ba852aa293f74f639369b6d224903aa1/roles/df8f9f844b1b439181d75a07c1a93418 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 16:11:25.027974 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3227e00-43cd-4452-b48b-a77e86a3995d None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:11:25.028254 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3227e00-43cd-4452-b48b-a77e86a3995d None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:11:25.028558 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3227e00-43cd-4452-b48b-a77e86a3995d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:11:25.048717 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:11:25.058027 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-a3227e00-43cd-4452-b48b-a77e86a3995d None None] MFA Rules not processed for user `ba852aa293f74f639369b6d224903aa1`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:11:25.059853 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-a3227e00-43cd-4452-b48b-a77e86a3995d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:11:25.126194 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1432/2889] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 16:11:25 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:11:25.137915 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77111c25-53a8-4456-90ad-926676bc5596 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:11:25.139696 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-77111c25-53a8-4456-90ad-926676bc5596 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:11:25.199042 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77111c25-53a8-4456-90ad-926676bc5596 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:11:25.203152 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56524b73-6579-47b6-9cb8-88f611816faf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0100a04821804993866ea20f159663f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:11:25.203754 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56524b73-6579-47b6-9cb8-88f611816faf service cinder] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:11:25.204031 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56524b73-6579-47b6-9cb8-88f611816faf service cinder] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:11:25.204298 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56524b73-6579-47b6-9cb8-88f611816faf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:11:25.212838 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56524b73-6579-47b6-9cb8-88f611816faf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:11:25.214332 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56524b73-6579-47b6-9cb8-88f611816faf service cinder] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:11:25.223747 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1458/2890] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 16:11:25 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:11:25.249785 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3227e00-43cd-4452-b48b-a77e86a3995d None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:11:25.314359 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3227e00-43cd-4452-b48b-a77e86a3995d None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:11:25.319159 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe093cd-6005-4ad3-addd-2ad0bf02ddbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:11:25.319787 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe093cd-6005-4ad3-addd-2ad0bf02ddbc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:11:25.320067 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe093cd-6005-4ad3-addd-2ad0bf02ddbc admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:11:25.320372 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe093cd-6005-4ad3-addd-2ad0bf02ddbc admin admin] PATH_INFO: `/v3/users/ba852aa293f74f639369b6d224903aa1` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:11:25.321584 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe093cd-6005-4ad3-addd-2ad0bf02ddbc admin admin] RBAC: Authorizing `identity:delete_user(user_id=ba852aa293f74f639369b6d224903aa1)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:11:25.323523 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe093cd-6005-4ad3-addd-2ad0bf02ddbc admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:11:25.374264 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-dbe093cd-6005-4ad3-addd-2ad0bf02ddbc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ba852aa293f74f639369b6d224903aa1'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:11:25.382708 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-dbe093cd-6005-4ad3-addd-2ad0bf02ddbc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ba852aa293f74f639369b6d224903aa1'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:11:25.397743 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-dbe093cd-6005-4ad3-addd-2ad0bf02ddbc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ba852aa293f74f639369b6d224903aa1'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:11:25.404537 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-dbe093cd-6005-4ad3-addd-2ad0bf02ddbc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ba852aa293f74f639369b6d224903aa1'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:11:25.405197 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-dbe093cd-6005-4ad3-addd-2ad0bf02ddbc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ba852aa293f74f639369b6d224903aa1'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:11:25.411372 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1433/2891] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Aug 30 16:11:25 2022] DELETE /identity/v3/users/ba852aa293f74f639369b6d224903aa1 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:11:25.415973 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75c7a1b-f3c8-47d9-8655-1d2ee2d0efd6 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:11:25.416257 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75c7a1b-f3c8-47d9-8655-1d2ee2d0efd6 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:11:25.416528 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75c7a1b-f3c8-47d9-8655-1d2ee2d0efd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:11:25.425506 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:11:25.440868 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-c75c7a1b-f3c8-47d9-8655-1d2ee2d0efd6 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:11:25.442179 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-c75c7a1b-f3c8-47d9-8655-1d2ee2d0efd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:11:25.505745 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1459/2892] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 16:11:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 16:11:25.515599 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe093cd-6005-4ad3-addd-2ad0bf02ddbc admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:11:25.517127 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-dbe093cd-6005-4ad3-addd-2ad0bf02ddbc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:11:25.537393 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:11:25.575834 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe093cd-6005-4ad3-addd-2ad0bf02ddbc admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:11:25.580196 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd015b1-7288-4130-9385-51af7a29e91a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f4157105f3488f807b7475008ab722', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:11:25.580818 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd015b1-7288-4130-9385-51af7a29e91a service neutron] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:11:25.581074 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd015b1-7288-4130-9385-51af7a29e91a service neutron] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:11:25.581376 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd015b1-7288-4130-9385-51af7a29e91a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:11:25.591527 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd015b1-7288-4130-9385-51af7a29e91a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:11:25.592959 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd015b1-7288-4130-9385-51af7a29e91a service neutron] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:11:25.602824 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1434/2893] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 16:11:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 16:11:25.901285 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75c7a1b-f3c8-47d9-8655-1d2ee2d0efd6 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:11:25.902889 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-c75c7a1b-f3c8-47d9-8655-1d2ee2d0efd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:11:25.919035 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75c7a1b-f3c8-47d9-8655-1d2ee2d0efd6 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:11:25.931469 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2774396-a013-42c5-bd5a-ecc594f329a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:11:25.932138 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2774396-a013-42c5-bd5a-ecc594f329a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:11:25.932403 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2774396-a013-42c5-bd5a-ecc594f329a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:11:25.932676 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2774396-a013-42c5-bd5a-ecc594f329a1 admin admin] PATH_INFO: `/v3/projects/1adbee870d654476a06ae02744a7ed95` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:11:25.933875 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2774396-a013-42c5-bd5a-ecc594f329a1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1adbee870d654476a06ae02744a7ed95)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:11:25.935812 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2774396-a013-42c5-bd5a-ecc594f329a1 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:11:25.997845 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-f2774396-a013-42c5-bd5a-ecc594f329a1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1adbee870d654476a06ae02744a7ed95', 'request_id': 'req-f2774396-a013-42c5-bd5a-ecc594f329a1'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:11:26.006308 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-f2774396-a013-42c5-bd5a-ecc594f329a1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1adbee870d654476a06ae02744a7ed95', 'request_id': 'req-f2774396-a013-42c5-bd5a-ecc594f329a1'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:11:26.012888 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-f2774396-a013-42c5-bd5a-ecc594f329a1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1adbee870d654476a06ae02744a7ed95', 'request_id': 'req-f2774396-a013-42c5-bd5a-ecc594f329a1'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:11:26.023739 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-f2774396-a013-42c5-bd5a-ecc594f329a1 admin admin] The token cache is being invalidate because project 1adbee870d654476a06ae02744a7ed95 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81511) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 16:11:26.024026 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-f2774396-a013-42c5-bd5a-ecc594f329a1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:11:26.024730 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-f2774396-a013-42c5-bd5a-ecc594f329a1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:11:26.026317 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1460/2894] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Aug 30 16:11:25 2022] DELETE /identity/v3/projects/1adbee870d654476a06ae02744a7ed95 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:11:26.031635 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da32c687-5514-4b7a-b009-f97e95ca48ba None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:11:26.032147 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da32c687-5514-4b7a-b009-f97e95ca48ba None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:11:26.032147 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da32c687-5514-4b7a-b009-f97e95ca48ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:11:26.052250 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-da32c687-5514-4b7a-b009-f97e95ca48ba None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:11:26.053472 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-da32c687-5514-4b7a-b009-f97e95ca48ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:11:26.115572 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1435/2895] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 16:11:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:11:26.123489 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0851c9da-a584-4223-9684-3f1087e91c03 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:11:26.123753 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0851c9da-a584-4223-9684-3f1087e91c03 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:11:26.124025 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0851c9da-a584-4223-9684-3f1087e91c03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:11:26.145475 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-0851c9da-a584-4223-9684-3f1087e91c03 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:11:26.146716 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-0851c9da-a584-4223-9684-3f1087e91c03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:11:26.157975 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1461/2896] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 16:11:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:11:26.162875 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da32c687-5514-4b7a-b009-f97e95ca48ba None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:11:26.173076 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da32c687-5514-4b7a-b009-f97e95ca48ba None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:11:26.178198 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d209ce-bc02-4079-8f71-a5003454161e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:11:26.178819 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d209ce-bc02-4079-8f71-a5003454161e admin admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:11:26.179096 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d209ce-bc02-4079-8f71-a5003454161e admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:11:26.179364 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d209ce-bc02-4079-8f71-a5003454161e admin admin] PATH_INFO: `/v3/domains` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:11:26.180041 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d209ce-bc02-4079-8f71-a5003454161e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81512) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:11:26.180342 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d209ce-bc02-4079-8f71-a5003454161e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:11:26.182062 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d209ce-bc02-4079-8f71-a5003454161e admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:11:26.193885 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1436/2897] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Aug 30 16:11:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:11:26.198614 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0851c9da-a584-4223-9684-3f1087e91c03 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:11:26.212688 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0851c9da-a584-4223-9684-3f1087e91c03 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:11:26.219430 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f59707-d030-4ced-9a2f-5c061e09a4cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:11:26.220068 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f59707-d030-4ced-9a2f-5c061e09a4cf admin admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:11:26.220339 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f59707-d030-4ced-9a2f-5c061e09a4cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:11:26.220603 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f59707-d030-4ced-9a2f-5c061e09a4cf admin admin] PATH_INFO: `/v3/projects` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:11:26.221260 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f59707-d030-4ced-9a2f-5c061e09a4cf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:11:26.224228 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f59707-d030-4ced-9a2f-5c061e09a4cf admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:11:26.254400 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1462/2898] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Aug 30 16:11:26 2022] POST /identity/v3/projects => generated 383 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:11:26.258754 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d209ce-bc02-4079-8f71-a5003454161e admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:11:26.269073 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d209ce-bc02-4079-8f71-a5003454161e admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:11:26.326460 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15af692-43b9-4783-ae21-2ace31b681f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:11:26.327063 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15af692-43b9-4783-ae21-2ace31b681f6 admin admin] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:11:26.327314 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15af692-43b9-4783-ae21-2ace31b681f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:11:26.327611 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15af692-43b9-4783-ae21-2ace31b681f6 admin admin] PATH_INFO: `/v3/users` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:11:26.328231 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15af692-43b9-4783-ae21-2ace31b681f6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:11:26.330181 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15af692-43b9-4783-ae21-2ace31b681f6 admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:11:26.357062 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1437/2899] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Aug 30 16:11:26 2022] POST /identity/v3/users => generated 292 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:11:26.363436 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f59707-d030-4ced-9a2f-5c061e09a4cf admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:11:26.374832 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f59707-d030-4ced-9a2f-5c061e09a4cf admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:11:26.379803 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bede0e-18f5-4b3d-a96b-9bd48af6df7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:11:26.380413 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bede0e-18f5-4b3d-a96b-9bd48af6df7c admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:11:26.380808 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bede0e-18f5-4b3d-a96b-9bd48af6df7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:11:26.380987 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bede0e-18f5-4b3d-a96b-9bd48af6df7c admin admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:11:26.381665 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7bede0e-18f5-4b3d-a96b-9bd48af6df7c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:11:26.383267 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7bede0e-18f5-4b3d-a96b-9bd48af6df7c admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:11:26.393197 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1463/2900] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 30 16:11:26 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:11:26.397451 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15af692-43b9-4783-ae21-2ace31b681f6 admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:11:26.407651 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15af692-43b9-4783-ae21-2ace31b681f6 admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:11:26.413480 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88355261-f0ae-4f55-9fb9-927d14e7c506 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:11:26.414078 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88355261-f0ae-4f55-9fb9-927d14e7c506 admin admin] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:11:26.414357 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88355261-f0ae-4f55-9fb9-927d14e7c506 admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:11:26.414624 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88355261-f0ae-4f55-9fb9-927d14e7c506 admin admin] PATH_INFO: `/v3/roles` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:11:26.415264 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88355261-f0ae-4f55-9fb9-927d14e7c506 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:11:26.416903 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88355261-f0ae-4f55-9fb9-927d14e7c506 admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:11:26.430914 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1438/2901] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 30 16:11:26 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:11:26.435288 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bede0e-18f5-4b3d-a96b-9bd48af6df7c admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:11:26.445503 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bede0e-18f5-4b3d-a96b-9bd48af6df7c admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:11:26.450577 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fb529f-7f85-435f-9bbd-258a3e02e4bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:11:26.451223 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fb529f-7f85-435f-9bbd-258a3e02e4bb admin admin] REQUEST_METHOD: `PUT` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:11:26.451503 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fb529f-7f85-435f-9bbd-258a3e02e4bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:11:26.451779 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fb529f-7f85-435f-9bbd-258a3e02e4bb admin admin] PATH_INFO: `/v3/projects/298e354e744742a3919e3dd321c84d4d/users/c5abe1348966489daba0656972760f65/roles/df8f9f844b1b439181d75a07c1a93418` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:11:26.466883 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9fb529f-7f85-435f-9bbd-258a3e02e4bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=298e354e744742a3919e3dd321c84d4d, user_id=c5abe1348966489daba0656972760f65, role_id=df8f9f844b1b439181d75a07c1a93418)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:11:26.469588 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9fb529f-7f85-435f-9bbd-258a3e02e4bb admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:11:26.485150 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1464/2902] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 30 16:11:26 2022] PUT /identity/v3/projects/298e354e744742a3919e3dd321c84d4d/users/c5abe1348966489daba0656972760f65/roles/df8f9f844b1b439181d75a07c1a93418 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:11:26.490307 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bb24d7-4ef9-4700-8bc7-5a71a4a3b231 None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:11:26.490641 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bb24d7-4ef9-4700-8bc7-5a71a4a3b231 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:11:26.490971 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bb24d7-4ef9-4700-8bc7-5a71a4a3b231 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:11:26.510312 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-c8bb24d7-4ef9-4700-8bc7-5a71a4a3b231 None None] MFA Rules not processed for user `8b9160d3cbae47a48dc9dee882da999b`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:11:26.511490 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-c8bb24d7-4ef9-4700-8bc7-5a71a4a3b231 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:11:26.532198 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:11:26.574644 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1439/2903] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 16:11:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:11:26.584721 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fb529f-7f85-435f-9bbd-258a3e02e4bb admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:11:26.586730 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-a9fb529f-7f85-435f-9bbd-258a3e02e4bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:11:26.614499 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:11:26.648403 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fb529f-7f85-435f-9bbd-258a3e02e4bb admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:11:26.652905 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad5c0fd-75de-49ef-bab9-018934b43454 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f4157105f3488f807b7475008ab722', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:11:26.653565 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad5c0fd-75de-49ef-bab9-018934b43454 service neutron] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:11:26.653846 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad5c0fd-75de-49ef-bab9-018934b43454 service neutron] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:11:26.654134 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad5c0fd-75de-49ef-bab9-018934b43454 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:11:26.664375 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad5c0fd-75de-49ef-bab9-018934b43454 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:11:26.665996 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad5c0fd-75de-49ef-bab9-018934b43454 service neutron] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:11:26.675727 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1465/2904] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 16:11:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:11:30.924135 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df74cae1-4675-4761-850c-76c238854c80 None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:11:30.924659 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df74cae1-4675-4761-850c-76c238854c80 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:11:30.924659 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df74cae1-4675-4761-850c-76c238854c80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:11:30.957459 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-df74cae1-4675-4761-850c-76c238854c80 None None] MFA Rules not processed for user `c5abe1348966489daba0656972760f65`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:11:30.958681 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-df74cae1-4675-4761-850c-76c238854c80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:11:31.040330 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1440/2905] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 16:11:30 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:11:31.045089 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad5c0fd-75de-49ef-bab9-018934b43454 service neutron] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:11:31.055129 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad5c0fd-75de-49ef-bab9-018934b43454 service neutron] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:11:31.059897 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d05ebd-05a1-4e54-bc15-aa83e30319f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:11:31.060500 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d05ebd-05a1-4e54-bc15-aa83e30319f4 admin admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:11:31.060782 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d05ebd-05a1-4e54-bc15-aa83e30319f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:11:31.061067 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d05ebd-05a1-4e54-bc15-aa83e30319f4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:11:31.061814 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d05ebd-05a1-4e54-bc15-aa83e30319f4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:11:31.063722 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d05ebd-05a1-4e54-bc15-aa83e30319f4 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:11:31.099473 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1466/2906] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Aug 30 16:11:31 2022] POST /identity/v3/projects => generated 381 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:11:31.100903 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df74cae1-4675-4761-850c-76c238854c80 None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:11:31.112104 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df74cae1-4675-4761-850c-76c238854c80 None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:11:31.172188 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5279704a-a904-43a1-afe3-1f273eafff68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:11:31.172804 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5279704a-a904-43a1-afe3-1f273eafff68 admin admin] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:11:31.173236 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5279704a-a904-43a1-afe3-1f273eafff68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:11:31.173518 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5279704a-a904-43a1-afe3-1f273eafff68 admin admin] PATH_INFO: `/v3/users` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:11:31.174156 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5279704a-a904-43a1-afe3-1f273eafff68 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:11:31.176349 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5279704a-a904-43a1-afe3-1f273eafff68 admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:11:31.198847 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1441/2907] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Aug 30 16:11:31 2022] POST /identity/v3/users => generated 297 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:11:31.202786 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d05ebd-05a1-4e54-bc15-aa83e30319f4 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:11:31.213748 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d05ebd-05a1-4e54-bc15-aa83e30319f4 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:11:31.218526 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646676ba-0a7a-4170-a876-5eaffc76c202 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:11:31.219092 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646676ba-0a7a-4170-a876-5eaffc76c202 admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:11:31.219374 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646676ba-0a7a-4170-a876-5eaffc76c202 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:11:31.219646 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646676ba-0a7a-4170-a876-5eaffc76c202 admin admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:11:31.220298 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646676ba-0a7a-4170-a876-5eaffc76c202 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:11:31.221950 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646676ba-0a7a-4170-a876-5eaffc76c202 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:11:31.235582 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1467/2908] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 30 16:11:31 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:11:31.239620 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5279704a-a904-43a1-afe3-1f273eafff68 admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:11:31.250288 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5279704a-a904-43a1-afe3-1f273eafff68 admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:11:31.255239 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b99e0e-f472-4069-b724-1333a4307b6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:11:31.255909 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b99e0e-f472-4069-b724-1333a4307b6f admin admin] REQUEST_METHOD: `PUT` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:11:31.256189 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b99e0e-f472-4069-b724-1333a4307b6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:11:31.256448 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b99e0e-f472-4069-b724-1333a4307b6f admin admin] PATH_INFO: `/v3/projects/08ac9d40c67949a4bccd8db0ba89936c/users/6a8f059e3b3a4388ac12b914d6959588/roles/f265386950a641ac8413b53eda4ae34d` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:11:31.272436 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b99e0e-f472-4069-b724-1333a4307b6f admin admin] RBAC: Authorizing `identity:create_grant(project_id=08ac9d40c67949a4bccd8db0ba89936c, user_id=6a8f059e3b3a4388ac12b914d6959588, role_id=f265386950a641ac8413b53eda4ae34d)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:11:31.275160 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b99e0e-f472-4069-b724-1333a4307b6f admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:11:31.290872 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1442/2909] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 30 16:11:31 2022] PUT /identity/v3/projects/08ac9d40c67949a4bccd8db0ba89936c/users/6a8f059e3b3a4388ac12b914d6959588/roles/f265386950a641ac8413b53eda4ae34d => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 16:11:31.294869 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646676ba-0a7a-4170-a876-5eaffc76c202 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:11:31.305280 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646676ba-0a7a-4170-a876-5eaffc76c202 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:11:31.363159 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c699c827-6c41-4b01-9638-c67d1ab97406 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:11:31.363811 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c699c827-6c41-4b01-9638-c67d1ab97406 admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:11:31.364084 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c699c827-6c41-4b01-9638-c67d1ab97406 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:11:31.364351 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c699c827-6c41-4b01-9638-c67d1ab97406 admin admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:11:31.365003 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c699c827-6c41-4b01-9638-c67d1ab97406 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:11:31.366641 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c699c827-6c41-4b01-9638-c67d1ab97406 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:11:31.377387 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1468/2910] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 30 16:11:31 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:11:31.381992 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b99e0e-f472-4069-b724-1333a4307b6f admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:11:31.392929 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b99e0e-f472-4069-b724-1333a4307b6f admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:11:31.398832 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddc4d7f-2c32-4403-b151-bad0b9347f11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:11:31.399521 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddc4d7f-2c32-4403-b151-bad0b9347f11 admin admin] REQUEST_METHOD: `PUT` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:11:31.399782 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddc4d7f-2c32-4403-b151-bad0b9347f11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:11:31.400051 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddc4d7f-2c32-4403-b151-bad0b9347f11 admin admin] PATH_INFO: `/v3/projects/08ac9d40c67949a4bccd8db0ba89936c/users/6a8f059e3b3a4388ac12b914d6959588/roles/f265386950a641ac8413b53eda4ae34d` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:11:31.402038 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ddc4d7f-2c32-4403-b151-bad0b9347f11 admin admin] RBAC: Authorizing `identity:create_grant(project_id=08ac9d40c67949a4bccd8db0ba89936c, user_id=6a8f059e3b3a4388ac12b914d6959588, role_id=f265386950a641ac8413b53eda4ae34d)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:11:31.404743 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ddc4d7f-2c32-4403-b151-bad0b9347f11 admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:11:31.415896 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1443/2911] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 30 16:11:31 2022] PUT /identity/v3/projects/08ac9d40c67949a4bccd8db0ba89936c/users/6a8f059e3b3a4388ac12b914d6959588/roles/f265386950a641ac8413b53eda4ae34d => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:11:35.861329 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75c0daf-4d88-4a72-8afc-11d084dc2be1 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:11:35.861912 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75c0daf-4d88-4a72-8afc-11d084dc2be1 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:11:35.862015 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75c0daf-4d88-4a72-8afc-11d084dc2be1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:11:35.893812 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-f75c0daf-4d88-4a72-8afc-11d084dc2be1 None None] MFA Rules not processed for user `6a8f059e3b3a4388ac12b914d6959588`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:11:35.895181 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-f75c0daf-4d88-4a72-8afc-11d084dc2be1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:11:35.972695 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1469/2912] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 16:11:35 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:11:35.985595 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddc4d7f-2c32-4403-b151-bad0b9347f11 admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:11:35.987385 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-0ddc4d7f-2c32-4403-b151-bad0b9347f11 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:11:36.001767 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:11:36.073578 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddc4d7f-2c32-4403-b151-bad0b9347f11 admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:11:36.077149 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3566ad07-eacf-43bc-8e4e-0afefc189d7d service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b70e06a7d06d4199ab4fc36342ee1d68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:11:36.077814 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3566ad07-eacf-43bc-8e4e-0afefc189d7d service glance] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:11:36.078066 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3566ad07-eacf-43bc-8e4e-0afefc189d7d service glance] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:11:36.078327 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3566ad07-eacf-43bc-8e4e-0afefc189d7d service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:11:36.138759 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3566ad07-eacf-43bc-8e4e-0afefc189d7d service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:11:36.140215 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3566ad07-eacf-43bc-8e4e-0afefc189d7d service glance] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:11:36.149980 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1444/2913] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 16:11:35 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:11:36.298063 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17f9084-5f75-4716-84e7-87793aee3fb2 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:11:36.298354 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17f9084-5f75-4716-84e7-87793aee3fb2 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:11:36.298626 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17f9084-5f75-4716-84e7-87793aee3fb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:11:36.318037 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:11:36.360399 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-e17f9084-5f75-4716-84e7-87793aee3fb2 None None] MFA Rules not processed for user `07130d598131443486c2bc1c6673c45b`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:11:36.361661 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-e17f9084-5f75-4716-84e7-87793aee3fb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:11:36.432963 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1470/2914] 10.222.0.20 () {58 vars in 1009 bytes} [Tue Aug 30 16:11:36 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:11:36.495379 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3566ad07-eacf-43bc-8e4e-0afefc189d7d service glance] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:11:36.497802 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-3566ad07-eacf-43bc-8e4e-0afefc189d7d service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:11:36.605106 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3566ad07-eacf-43bc-8e4e-0afefc189d7d service glance] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:11:36.612575 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c41b06-47d4-4145-88f2-151bca766647 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8fdafdc39a594bb4b0aeaf2fbc84c2fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:11:36.613191 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c41b06-47d4-4145-88f2-151bca766647 service swift] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:11:36.613499 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c41b06-47d4-4145-88f2-151bca766647 service swift] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:11:36.613766 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c41b06-47d4-4145-88f2-151bca766647 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:11:36.622589 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c41b06-47d4-4145-88f2-151bca766647 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:11:36.624113 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c41b06-47d4-4145-88f2-151bca766647 service swift] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:11:36.634414 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1445/2915] 10.222.0.20 () {60 vars in 1384 bytes} [Tue Aug 30 16:11:36 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 30 16:11:44.962016 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ed672a-a9ee-4dbe-b1f5-738fc4c37381 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:11:44.962566 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ed672a-a9ee-4dbe-b1f5-738fc4c37381 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:11:44.962566 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ed672a-a9ee-4dbe-b1f5-738fc4c37381 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:11:44.983507 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-69ed672a-a9ee-4dbe-b1f5-738fc4c37381 None None] MFA Rules not processed for user `c5abe1348966489daba0656972760f65`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:11:44.984742 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-69ed672a-a9ee-4dbe-b1f5-738fc4c37381 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:11:44.993012 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1471/2916] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 16:11:44 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:11:46.629711 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=94 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:11:47.044173 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94040e2-6c8d-41bb-928d-919f1988ff82 None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:11:47.044448 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94040e2-6c8d-41bb-928d-919f1988ff82 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:11:47.044722 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94040e2-6c8d-41bb-928d-919f1988ff82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:11:47.064441 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-e94040e2-6c8d-41bb-928d-919f1988ff82 None None] MFA Rules not processed for user `07130d598131443486c2bc1c6673c45b`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:11:47.065726 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-e94040e2-6c8d-41bb-928d-919f1988ff82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:11:47.072905 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1446/2917] 10.222.0.20 () {58 vars in 1009 bytes} [Tue Aug 30 16:11:47 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:11:47.085183 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ed672a-a9ee-4dbe-b1f5-738fc4c37381 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:11:47.097769 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ed672a-a9ee-4dbe-b1f5-738fc4c37381 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:11:47.103225 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7d4e87-14c2-4f9c-96b8-debb4a6b8237 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8fdafdc39a594bb4b0aeaf2fbc84c2fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:11:47.103846 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7d4e87-14c2-4f9c-96b8-debb4a6b8237 service swift] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:11:47.104122 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7d4e87-14c2-4f9c-96b8-debb4a6b8237 service swift] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:11:47.104390 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7d4e87-14c2-4f9c-96b8-debb4a6b8237 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:11:47.113084 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7d4e87-14c2-4f9c-96b8-debb4a6b8237 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:11:47.114738 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7d4e87-14c2-4f9c-96b8-debb4a6b8237 service swift] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:11:47.123188 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1472/2918] 10.222.0.20 () {60 vars in 1384 bytes} [Tue Aug 30 16:11:47 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 30 16:11:57.059461 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=3 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:11:57.120059 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=47 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:12:14.950001 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94040e2-6c8d-41bb-928d-919f1988ff82 None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:12:15.016376 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94040e2-6c8d-41bb-928d-919f1988ff82 None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:12:15.021030 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d469856-506a-4c5f-bcd9-d4ef4e393169 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f4157105f3488f807b7475008ab722', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:12:15.021671 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d469856-506a-4c5f-bcd9-d4ef4e393169 service neutron] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:12:15.021946 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d469856-506a-4c5f-bcd9-d4ef4e393169 service neutron] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:12:15.022281 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d469856-506a-4c5f-bcd9-d4ef4e393169 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:12:15.032244 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d469856-506a-4c5f-bcd9-d4ef4e393169 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:12:15.033728 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d469856-506a-4c5f-bcd9-d4ef4e393169 service neutron] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:12:15.043241 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1447/2919] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 16:12:14 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:12:16.922887 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7d4e87-14c2-4f9c-96b8-debb4a6b8237 service swift] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:12:16.925287 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-7a7d4e87-14c2-4f9c-96b8-debb4a6b8237 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:12:17.009164 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7d4e87-14c2-4f9c-96b8-debb4a6b8237 service swift] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:12:17.015587 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81d721f-c78b-42da-8696-a4e515ba4ce6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a1e756ae9be4a7bac1e81acf07e8f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:12:17.016364 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81d721f-c78b-42da-8696-a4e515ba4ce6 service nova] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:12:17.016652 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81d721f-c78b-42da-8696-a4e515ba4ce6 service nova] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:12:17.016920 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81d721f-c78b-42da-8696-a4e515ba4ce6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:12:17.018802 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-a81d721f-c78b-42da-8696-a4e515ba4ce6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:12:17.032148 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81d721f-c78b-42da-8696-a4e515ba4ce6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:12:17.033620 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81d721f-c78b-42da-8696-a4e515ba4ce6 service nova] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:12:17.045196 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1473/2920] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 16:12:16 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:12:25.039232 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=26 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:12:27.041207 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=34 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:12:46.118179 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7785b90f-f97d-49db-8632-ab71c8e33c76 None None] REQUEST_METHOD: `POST` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:12:46.118719 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7785b90f-f97d-49db-8632-ab71c8e33c76 None None] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:12:46.118719 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7785b90f-f97d-49db-8632-ab71c8e33c76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:12:46.146415 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.auth.core [None req-7785b90f-f97d-49db-8632-ab71c8e33c76 None None] MFA Rules not processed for user `02287f232cdd4842a747bb749b483cc3`. Rule list: `[]` (Enabled: `True`). {{(pid=81512) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:12:46.148140 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-7785b90f-f97d-49db-8632-ab71c8e33c76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:12:46.208412 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1448/2921] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 30 16:12:46 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:12:46.554111 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81d721f-c78b-42da-8696-a4e515ba4ce6 service nova] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:12:46.555955 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-a81d721f-c78b-42da-8696-a4e515ba4ce6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:12:46.568667 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81d721f-c78b-42da-8696-a4e515ba4ce6 service nova] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:12:46.572746 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22dcc501-c9a6-4159-82ce-c6e24b266894 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f4157105f3488f807b7475008ab722', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:12:46.573387 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dcc501-c9a6-4159-82ce-c6e24b266894 service neutron] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:12:46.573660 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dcc501-c9a6-4159-82ce-c6e24b266894 service neutron] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:12:46.573928 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dcc501-c9a6-4159-82ce-c6e24b266894 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:12:46.575794 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-22dcc501-c9a6-4159-82ce-c6e24b266894 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:12:46.585707 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22dcc501-c9a6-4159-82ce-c6e24b266894 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:12:46.587124 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22dcc501-c9a6-4159-82ce-c6e24b266894 service neutron] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:12:46.595151 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1474/2922] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 16:12:46 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:12:47.602378 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7785b90f-f97d-49db-8632-ab71c8e33c76 None None] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:12:47.604013 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-7785b90f-f97d-49db-8632-ab71c8e33c76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:12:47.679341 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7785b90f-f97d-49db-8632-ab71c8e33c76 None None] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:12:47.685354 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0384247e-170e-4b24-8f35-12ce29948676 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc12c6d31eab4a17a160250adc2380c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:12:47.685971 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0384247e-170e-4b24-8f35-12ce29948676 service placement] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:12:47.686255 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0384247e-170e-4b24-8f35-12ce29948676 service placement] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:12:47.686530 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0384247e-170e-4b24-8f35-12ce29948676 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:12:47.688404 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-0384247e-170e-4b24-8f35-12ce29948676 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:12:47.700342 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0384247e-170e-4b24-8f35-12ce29948676 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:12:47.701973 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0384247e-170e-4b24-8f35-12ce29948676 service placement] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:12:47.712175 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1449/2923] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 16:12:47 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:12:56.591854 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=6 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:12:57.707579 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=57 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:13:30.379292 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e0ef5c-7054-483a-8d95-c91f53e21b0e None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:13:30.379794 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e0ef5c-7054-483a-8d95-c91f53e21b0e None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:13:30.379901 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e0ef5c-7054-483a-8d95-c91f53e21b0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:13:30.402672 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-02e0ef5c-7054-483a-8d95-c91f53e21b0e None None] MFA Rules not processed for user `07130d598131443486c2bc1c6673c45b`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:13:30.403819 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-02e0ef5c-7054-483a-8d95-c91f53e21b0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:13:30.412350 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1475/2924] 10.222.0.20 () {58 vars in 967 bytes} [Tue Aug 30 16:13:30 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:13:30.453572 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0384247e-170e-4b24-8f35-12ce29948676 service placement] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:13:30.465435 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0384247e-170e-4b24-8f35-12ce29948676 service placement] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:13:30.471126 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0e610a-2775-4d88-ae61-a253c7b54928 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8fdafdc39a594bb4b0aeaf2fbc84c2fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:13:30.471725 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0e610a-2775-4d88-ae61-a253c7b54928 service swift] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:13:30.472024 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0e610a-2775-4d88-ae61-a253c7b54928 service swift] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:13:30.472278 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0e610a-2775-4d88-ae61-a253c7b54928 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:13:30.480577 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0e610a-2775-4d88-ae61-a253c7b54928 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:13:30.482207 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0e610a-2775-4d88-ae61-a253c7b54928 service swift] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:13:30.490651 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1450/2925] 10.222.0.20 () {60 vars in 1384 bytes} [Tue Aug 30 16:13:30 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 30 16:13:33.962882 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e0ef5c-7054-483a-8d95-c91f53e21b0e None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:13:33.982446 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e0ef5c-7054-483a-8d95-c91f53e21b0e None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:13:33.987760 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b7a290-6520-4c76-899d-6334113c7364 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc12c6d31eab4a17a160250adc2380c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:13:33.988509 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b7a290-6520-4c76-899d-6334113c7364 service placement] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:13:33.988791 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b7a290-6520-4c76-899d-6334113c7364 service placement] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:13:33.989061 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b7a290-6520-4c76-899d-6334113c7364 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:13:33.991324 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-09b7a290-6520-4c76-899d-6334113c7364 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:13:34.003859 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b7a290-6520-4c76-899d-6334113c7364 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:13:34.005584 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b7a290-6520-4c76-899d-6334113c7364 service placement] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:13:34.020207 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1476/2926] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 16:13:33 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:13:37.013149 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0e610a-2775-4d88-ae61-a253c7b54928 service swift] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:13:37.025974 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0e610a-2775-4d88-ae61-a253c7b54928 service swift] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:13:37.043757 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd6d987-8cfb-4ea7-bbe4-acbbb33be660 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc12c6d31eab4a17a160250adc2380c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:13:37.044449 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd6d987-8cfb-4ea7-bbe4-acbbb33be660 service placement] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:13:37.044976 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd6d987-8cfb-4ea7-bbe4-acbbb33be660 service placement] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:13:37.045179 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd6d987-8cfb-4ea7-bbe4-acbbb33be660 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:13:37.047297 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-3cd6d987-8cfb-4ea7-bbe4-acbbb33be660 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:13:37.060527 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd6d987-8cfb-4ea7-bbe4-acbbb33be660 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:13:37.063243 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd6d987-8cfb-4ea7-bbe4-acbbb33be660 service placement] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:13:37.075489 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1451/2927] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 16:13:37 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:13:37.092518 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b7a290-6520-4c76-899d-6334113c7364 service placement] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:13:37.159274 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b7a290-6520-4c76-899d-6334113c7364 service placement] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:13:37.163658 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d7ef57-1fa2-4c0b-8ac8-d12e3859dee7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:13:37.164285 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d7ef57-1fa2-4c0b-8ac8-d12e3859dee7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:13:37.164557 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d7ef57-1fa2-4c0b-8ac8-d12e3859dee7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:13:37.164948 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d7ef57-1fa2-4c0b-8ac8-d12e3859dee7 admin admin] PATH_INFO: `/v3/users/c5abe1348966489daba0656972760f65` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:13:37.166081 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d7ef57-1fa2-4c0b-8ac8-d12e3859dee7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c5abe1348966489daba0656972760f65)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:13:37.168000 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d7ef57-1fa2-4c0b-8ac8-d12e3859dee7 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:13:37.216192 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-f3d7ef57-1fa2-4c0b-8ac8-d12e3859dee7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c5abe1348966489daba0656972760f65'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:13:37.229444 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-f3d7ef57-1fa2-4c0b-8ac8-d12e3859dee7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c5abe1348966489daba0656972760f65'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:13:37.237764 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-f3d7ef57-1fa2-4c0b-8ac8-d12e3859dee7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c5abe1348966489daba0656972760f65'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:13:37.238291 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-f3d7ef57-1fa2-4c0b-8ac8-d12e3859dee7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c5abe1348966489daba0656972760f65'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:13:37.245725 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-f3d7ef57-1fa2-4c0b-8ac8-d12e3859dee7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c5abe1348966489daba0656972760f65'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:13:37.251766 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1477/2928] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Aug 30 16:13:37 2022] DELETE /identity/v3/users/c5abe1348966489daba0656972760f65 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:13:37.436160 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd6d987-8cfb-4ea7-bbe4-acbbb33be660 service placement] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:13:37.453268 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd6d987-8cfb-4ea7-bbe4-acbbb33be660 service placement] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:13:37.511842 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0066f45d-9777-4090-89f9-6207ec37c813 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:13:37.512393 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0066f45d-9777-4090-89f9-6207ec37c813 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:13:37.512677 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0066f45d-9777-4090-89f9-6207ec37c813 admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:13:37.512979 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0066f45d-9777-4090-89f9-6207ec37c813 admin admin] PATH_INFO: `/v3/projects/298e354e744742a3919e3dd321c84d4d` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:13:37.514138 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0066f45d-9777-4090-89f9-6207ec37c813 admin admin] RBAC: Authorizing `identity:delete_project(project_id=298e354e744742a3919e3dd321c84d4d)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:13:37.516085 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0066f45d-9777-4090-89f9-6207ec37c813 admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:13:37.564098 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-0066f45d-9777-4090-89f9-6207ec37c813 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '298e354e744742a3919e3dd321c84d4d', 'request_id': 'req-0066f45d-9777-4090-89f9-6207ec37c813'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:13:37.572634 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-0066f45d-9777-4090-89f9-6207ec37c813 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '298e354e744742a3919e3dd321c84d4d', 'request_id': 'req-0066f45d-9777-4090-89f9-6207ec37c813'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:13:37.579179 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-0066f45d-9777-4090-89f9-6207ec37c813 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '298e354e744742a3919e3dd321c84d4d', 'request_id': 'req-0066f45d-9777-4090-89f9-6207ec37c813'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:13:37.590491 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-0066f45d-9777-4090-89f9-6207ec37c813 admin admin] The token cache is being invalidate because project 298e354e744742a3919e3dd321c84d4d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81512) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 16:13:37.590776 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-0066f45d-9777-4090-89f9-6207ec37c813 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:13:37.591728 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-0066f45d-9777-4090-89f9-6207ec37c813 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:13:37.593435 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1452/2929] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Aug 30 16:13:37 2022] DELETE /identity/v3/projects/298e354e744742a3919e3dd321c84d4d => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:13:37.598067 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d7ef57-1fa2-4c0b-8ac8-d12e3859dee7 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:13:37.599742 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-f3d7ef57-1fa2-4c0b-8ac8-d12e3859dee7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:13:37.613437 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d7ef57-1fa2-4c0b-8ac8-d12e3859dee7 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:13:37.680616 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093503c2-110b-4477-9cda-1974758a4db9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:13:37.681277 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093503c2-110b-4477-9cda-1974758a4db9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:13:37.681554 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093503c2-110b-4477-9cda-1974758a4db9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:13:37.681824 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093503c2-110b-4477-9cda-1974758a4db9 admin admin] PATH_INFO: `/v3/users/6a8f059e3b3a4388ac12b914d6959588` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:13:37.682984 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093503c2-110b-4477-9cda-1974758a4db9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6a8f059e3b3a4388ac12b914d6959588)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:13:37.684956 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093503c2-110b-4477-9cda-1974758a4db9 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:13:37.686489 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:13:37.742699 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-093503c2-110b-4477-9cda-1974758a4db9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6a8f059e3b3a4388ac12b914d6959588'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:13:37.755005 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-093503c2-110b-4477-9cda-1974758a4db9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6a8f059e3b3a4388ac12b914d6959588'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:13:37.764119 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-093503c2-110b-4477-9cda-1974758a4db9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6a8f059e3b3a4388ac12b914d6959588'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:13:37.764721 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-093503c2-110b-4477-9cda-1974758a4db9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6a8f059e3b3a4388ac12b914d6959588'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:13:37.772208 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-093503c2-110b-4477-9cda-1974758a4db9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6a8f059e3b3a4388ac12b914d6959588'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:13:37.778359 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1478/2930] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Aug 30 16:13:37 2022] DELETE /identity/v3/users/6a8f059e3b3a4388ac12b914d6959588 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:13:37.966324 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0066f45d-9777-4090-89f9-6207ec37c813 admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:13:37.967800 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-0066f45d-9777-4090-89f9-6207ec37c813 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:13:37.983681 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0066f45d-9777-4090-89f9-6207ec37c813 admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:13:38.044841 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99eca063-f8a5-4f0f-bb60-3c3580242899 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:13:38.045523 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99eca063-f8a5-4f0f-bb60-3c3580242899 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:13:38.045748 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99eca063-f8a5-4f0f-bb60-3c3580242899 admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:13:38.045931 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99eca063-f8a5-4f0f-bb60-3c3580242899 admin admin] PATH_INFO: `/v3/projects/08ac9d40c67949a4bccd8db0ba89936c` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:13:38.047161 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99eca063-f8a5-4f0f-bb60-3c3580242899 admin admin] RBAC: Authorizing `identity:delete_project(project_id=08ac9d40c67949a4bccd8db0ba89936c)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:13:38.049083 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99eca063-f8a5-4f0f-bb60-3c3580242899 admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:13:38.055918 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:13:38.101402 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-99eca063-f8a5-4f0f-bb60-3c3580242899 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '08ac9d40c67949a4bccd8db0ba89936c', 'request_id': 'req-99eca063-f8a5-4f0f-bb60-3c3580242899'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:13:38.110362 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-99eca063-f8a5-4f0f-bb60-3c3580242899 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '08ac9d40c67949a4bccd8db0ba89936c', 'request_id': 'req-99eca063-f8a5-4f0f-bb60-3c3580242899'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:13:38.116859 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-99eca063-f8a5-4f0f-bb60-3c3580242899 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '08ac9d40c67949a4bccd8db0ba89936c', 'request_id': 'req-99eca063-f8a5-4f0f-bb60-3c3580242899'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:13:38.127292 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-99eca063-f8a5-4f0f-bb60-3c3580242899 admin admin] The token cache is being invalidate because project 08ac9d40c67949a4bccd8db0ba89936c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81512) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 16:13:38.127581 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-99eca063-f8a5-4f0f-bb60-3c3580242899 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:13:38.128229 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-99eca063-f8a5-4f0f-bb60-3c3580242899 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:13:38.129790 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1453/2931] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Aug 30 16:13:37 2022] DELETE /identity/v3/projects/08ac9d40c67949a4bccd8db0ba89936c => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:13:47.774730 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=26,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:13:48.124256 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=22,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:14:35.834885 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093503c2-110b-4477-9cda-1974758a4db9 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:14:35.836986 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-093503c2-110b-4477-9cda-1974758a4db9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:14:35.915689 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093503c2-110b-4477-9cda-1974758a4db9 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:14:35.971984 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40d2a26-aeb4-4ab0-8b98-6e8b87654e4c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f4157105f3488f807b7475008ab722', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:14:35.972657 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40d2a26-aeb4-4ab0-8b98-6e8b87654e4c service neutron] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:14:35.972902 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40d2a26-aeb4-4ab0-8b98-6e8b87654e4c service neutron] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:14:35.973259 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40d2a26-aeb4-4ab0-8b98-6e8b87654e4c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:14:35.975233 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-a40d2a26-aeb4-4ab0-8b98-6e8b87654e4c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:14:36.051039 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40d2a26-aeb4-4ab0-8b98-6e8b87654e4c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:14:36.053166 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40d2a26-aeb4-4ab0-8b98-6e8b87654e4c service neutron] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:14:36.066875 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1479/2932] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 16:14:35 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:14:37.148500 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99eca063-f8a5-4f0f-bb60-3c3580242899 admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:14:37.150074 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-99eca063-f8a5-4f0f-bb60-3c3580242899 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:14:37.162260 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99eca063-f8a5-4f0f-bb60-3c3580242899 admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:14:37.166209 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ac8596-5d41-4f17-84ce-8ffd78ef9308 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f4157105f3488f807b7475008ab722', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:14:37.166810 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ac8596-5d41-4f17-84ce-8ffd78ef9308 service neutron] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:14:37.167083 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ac8596-5d41-4f17-84ce-8ffd78ef9308 service neutron] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:14:37.167347 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ac8596-5d41-4f17-84ce-8ffd78ef9308 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:14:37.169082 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-b4ac8596-5d41-4f17-84ce-8ffd78ef9308 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:14:37.245417 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ac8596-5d41-4f17-84ce-8ffd78ef9308 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:14:37.246957 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ac8596-5d41-4f17-84ce-8ffd78ef9308 service neutron] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:14:37.258679 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1454/2933] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 16:14:37 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:14:38.458501 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40d2a26-aeb4-4ab0-8b98-6e8b87654e4c service neutron] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:14:38.470251 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40d2a26-aeb4-4ab0-8b98-6e8b87654e4c service neutron] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:14:38.474813 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f0df0f-51fa-4e06-872b-aa8a40320c07 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f4157105f3488f807b7475008ab722', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:14:38.475507 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f0df0f-51fa-4e06-872b-aa8a40320c07 service neutron] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:14:38.475778 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f0df0f-51fa-4e06-872b-aa8a40320c07 service neutron] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:14:38.476041 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f0df0f-51fa-4e06-872b-aa8a40320c07 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:14:38.478046 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-09f0df0f-51fa-4e06-872b-aa8a40320c07 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:14:38.493837 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f0df0f-51fa-4e06-872b-aa8a40320c07 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:14:38.495259 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f0df0f-51fa-4e06-872b-aa8a40320c07 service neutron] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:14:38.506302 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1480/2934] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 16:14:38 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:14:47.253605 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=34 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:14:48.501698 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=74 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:14:50.449281 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ac8596-5d41-4f17-84ce-8ffd78ef9308 service neutron] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:14:50.459133 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ac8596-5d41-4f17-84ce-8ffd78ef9308 service neutron] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:14:50.463434 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb83314-9e1a-4cd2-a995-ad22f700f784 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f4157105f3488f807b7475008ab722', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:14:50.464041 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb83314-9e1a-4cd2-a995-ad22f700f784 service neutron] REQUEST_METHOD: `GET` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:14:50.464348 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb83314-9e1a-4cd2-a995-ad22f700f784 service neutron] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:14:50.464609 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb83314-9e1a-4cd2-a995-ad22f700f784 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:14:50.466712 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-edb83314-9e1a-4cd2-a995-ad22f700f784 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:14:50.531120 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb83314-9e1a-4cd2-a995-ad22f700f784 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:14:50.532608 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb83314-9e1a-4cd2-a995-ad22f700f784 service neutron] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:14:50.542675 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1455/2935] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 16:14:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:14:56.688118 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f0df0f-51fa-4e06-872b-aa8a40320c07 service neutron] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:14:56.690054 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-09f0df0f-51fa-4e06-872b-aa8a40320c07 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:14:56.704718 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f0df0f-51fa-4e06-872b-aa8a40320c07 service neutron] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:14:56.709606 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d6cf88-8368-493a-92cf-ce5f66a5fc61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:14:56.710261 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d6cf88-8368-493a-92cf-ce5f66a5fc61 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:14:56.710538 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d6cf88-8368-493a-92cf-ce5f66a5fc61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:14:56.710806 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d6cf88-8368-493a-92cf-ce5f66a5fc61 admin admin] PATH_INFO: `/v3/users/02287f232cdd4842a747bb749b483cc3` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:14:56.711968 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d6cf88-8368-493a-92cf-ce5f66a5fc61 admin admin] RBAC: Authorizing `identity:delete_user(user_id=02287f232cdd4842a747bb749b483cc3)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:14:56.713995 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d6cf88-8368-493a-92cf-ce5f66a5fc61 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:14:56.767036 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-43d6cf88-8368-493a-92cf-ce5f66a5fc61 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '02287f232cdd4842a747bb749b483cc3'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:14:56.780123 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-43d6cf88-8368-493a-92cf-ce5f66a5fc61 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '02287f232cdd4842a747bb749b483cc3'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:14:56.788711 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-43d6cf88-8368-493a-92cf-ce5f66a5fc61 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '02287f232cdd4842a747bb749b483cc3'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:14:56.789487 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-43d6cf88-8368-493a-92cf-ce5f66a5fc61 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '02287f232cdd4842a747bb749b483cc3'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:14:56.797085 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-43d6cf88-8368-493a-92cf-ce5f66a5fc61 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '02287f232cdd4842a747bb749b483cc3'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:14:56.803292 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1481/2936] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Aug 30 16:14:56 2022] DELETE /identity/v3/users/02287f232cdd4842a747bb749b483cc3 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 16:14:56.978561 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb83314-9e1a-4cd2-a995-ad22f700f784 service neutron] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:14:56.988750 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb83314-9e1a-4cd2-a995-ad22f700f784 service neutron] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:14:57.048148 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b3a8de-6451-41d1-9b62-99022ed5178a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:14:57.048777 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b3a8de-6451-41d1-9b62-99022ed5178a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:14:57.049052 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b3a8de-6451-41d1-9b62-99022ed5178a admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:14:57.049348 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b3a8de-6451-41d1-9b62-99022ed5178a admin admin] PATH_INFO: `/v3/projects/330e7c1cf5894ac9b6d21915c5f645bc` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:14:57.050492 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b3a8de-6451-41d1-9b62-99022ed5178a admin admin] RBAC: Authorizing `identity:delete_project(project_id=330e7c1cf5894ac9b6d21915c5f645bc)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:14:57.052558 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b3a8de-6451-41d1-9b62-99022ed5178a admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:14:57.101160 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-23b3a8de-6451-41d1-9b62-99022ed5178a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '330e7c1cf5894ac9b6d21915c5f645bc', 'request_id': 'req-23b3a8de-6451-41d1-9b62-99022ed5178a'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:14:57.108278 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-23b3a8de-6451-41d1-9b62-99022ed5178a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '330e7c1cf5894ac9b6d21915c5f645bc', 'request_id': 'req-23b3a8de-6451-41d1-9b62-99022ed5178a'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:14:57.114538 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-23b3a8de-6451-41d1-9b62-99022ed5178a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '330e7c1cf5894ac9b6d21915c5f645bc', 'request_id': 'req-23b3a8de-6451-41d1-9b62-99022ed5178a'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:14:57.125373 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-23b3a8de-6451-41d1-9b62-99022ed5178a admin admin] The token cache is being invalidate because project 330e7c1cf5894ac9b6d21915c5f645bc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81512) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 16:14:57.125665 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-23b3a8de-6451-41d1-9b62-99022ed5178a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:14:57.126316 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-23b3a8de-6451-41d1-9b62-99022ed5178a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:14:57.127828 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1456/2937] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Aug 30 16:14:56 2022] DELETE /identity/v3/projects/330e7c1cf5894ac9b6d21915c5f645bc => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:14:57.132414 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d6cf88-8368-493a-92cf-ce5f66a5fc61 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:14:57.134765 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-43d6cf88-8368-493a-92cf-ce5f66a5fc61 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:14:57.150970 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d6cf88-8368-493a-92cf-ce5f66a5fc61 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:14:57.212627 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f85132-9ce4-47ac-88d4-95e7d9a4dff9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:14:57.213258 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f85132-9ce4-47ac-88d4-95e7d9a4dff9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:14:57.213584 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f85132-9ce4-47ac-88d4-95e7d9a4dff9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:14:57.213857 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f85132-9ce4-47ac-88d4-95e7d9a4dff9 admin admin] PATH_INFO: `/v3/users/579b15c74c00457184dcd978b929dbbb` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:14:57.215021 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f85132-9ce4-47ac-88d4-95e7d9a4dff9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=579b15c74c00457184dcd978b929dbbb)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:14:57.217260 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f85132-9ce4-47ac-88d4-95e7d9a4dff9 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:14:57.264988 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-82f85132-9ce4-47ac-88d4-95e7d9a4dff9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '579b15c74c00457184dcd978b929dbbb'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:14:57.276345 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-82f85132-9ce4-47ac-88d4-95e7d9a4dff9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '579b15c74c00457184dcd978b929dbbb'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:14:57.285076 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-82f85132-9ce4-47ac-88d4-95e7d9a4dff9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '579b15c74c00457184dcd978b929dbbb'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:14:57.285739 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-82f85132-9ce4-47ac-88d4-95e7d9a4dff9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '579b15c74c00457184dcd978b929dbbb'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:14:57.288093 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:14:57.294074 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-82f85132-9ce4-47ac-88d4-95e7d9a4dff9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '579b15c74c00457184dcd978b929dbbb'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:14:57.301094 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1482/2938] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Aug 30 16:14:57 2022] DELETE /identity/v3/users/579b15c74c00457184dcd978b929dbbb => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:14:57.483588 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b3a8de-6451-41d1-9b62-99022ed5178a admin admin] Authenticating user token {{(pid=81512) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:14:57.485077 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.fernet_utils [None req-23b3a8de-6451-41d1-9b62-99022ed5178a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81512) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:14:57.502746 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b3a8de-6451-41d1-9b62-99022ed5178a admin admin] Validating token access rules against request {{(pid=81512) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:14:57.524407 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:14:57.562170 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60df4fdd-8413-4f3b-b31b-0dae7c44a56d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b9160d3cbae47a48dc9dee882da999b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6954fa5b9a94855887d7068fc788d39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81512) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:14:57.562805 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60df4fdd-8413-4f3b-b31b-0dae7c44a56d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:14:57.563077 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60df4fdd-8413-4f3b-b31b-0dae7c44a56d admin admin] SCRIPT_NAME: `/identity` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:14:57.563346 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60df4fdd-8413-4f3b-b31b-0dae7c44a56d admin admin] PATH_INFO: `/v3/projects/1231f24ce4ae4692b2fde5e2a63debef` {{(pid=81512) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:14:57.565536 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60df4fdd-8413-4f3b-b31b-0dae7c44a56d admin admin] RBAC: Authorizing `identity:delete_project(project_id=1231f24ce4ae4692b2fde5e2a63debef)` {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:14:57.566554 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60df4fdd-8413-4f3b-b31b-0dae7c44a56d admin admin] RBAC: Authorization granted {{(pid=81512) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:14:57.615997 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-60df4fdd-8413-4f3b-b31b-0dae7c44a56d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1231f24ce4ae4692b2fde5e2a63debef', 'request_id': 'req-60df4fdd-8413-4f3b-b31b-0dae7c44a56d'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:14:57.623877 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-60df4fdd-8413-4f3b-b31b-0dae7c44a56d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1231f24ce4ae4692b2fde5e2a63debef', 'request_id': 'req-60df4fdd-8413-4f3b-b31b-0dae7c44a56d'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:14:57.630388 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-60df4fdd-8413-4f3b-b31b-0dae7c44a56d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1231f24ce4ae4692b2fde5e2a63debef', 'request_id': 'req-60df4fdd-8413-4f3b-b31b-0dae7c44a56d'} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:14:57.641745 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-60df4fdd-8413-4f3b-b31b-0dae7c44a56d admin admin] The token cache is being invalidate because project 1231f24ce4ae4692b2fde5e2a63debef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81512) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 16:14:57.642029 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-60df4fdd-8413-4f3b-b31b-0dae7c44a56d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:14:57.642714 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG keystone.notifications [None req-60df4fdd-8413-4f3b-b31b-0dae7c44a56d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81512) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:14:57.644333 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: [pid: 81512|app: 0|req: 1457/2939] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Aug 30 16:14:57 2022] DELETE /identity/v3/projects/1231f24ce4ae4692b2fde5e2a63debef => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 16:15:07.299449 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=4 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:15:07.638358 dv-cinder-855058-1-iscsi devstack@keystone.service[81512]: DEBUG dbcounter [-] [81512] Writing DB stats keystone:SELECT=40,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81512) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:18:36.124854 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f85132-9ce4-47ac-88d4-95e7d9a4dff9 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:18:36.126590 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-82f85132-9ce4-47ac-88d4-95e7d9a4dff9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:18:36.278452 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f85132-9ce4-47ac-88d4-95e7d9a4dff9 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:18:36.339736 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301aff00-2a12-43af-8f1c-819bdc89497b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc12c6d31eab4a17a160250adc2380c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '854fe71b49224dff9a4b1766a20ab266', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:18:36.340342 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301aff00-2a12-43af-8f1c-819bdc89497b service placement] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:18:36.340617 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301aff00-2a12-43af-8f1c-819bdc89497b service placement] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:18:36.340990 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301aff00-2a12-43af-8f1c-819bdc89497b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:18:36.342959 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-301aff00-2a12-43af-8f1c-819bdc89497b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:18:36.354699 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301aff00-2a12-43af-8f1c-819bdc89497b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:18:36.356312 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301aff00-2a12-43af-8f1c-819bdc89497b service placement] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:18:36.366976 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1483/2940] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 30 16:18:36 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 243 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:18:46.362593 dv-cinder-855058-1-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=89 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}